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FOREWORD 


These proceedings contain the papers of the International Conference e-Learning 2014, 
which was organised by the International Association for Development of the Information 
Society, 15-18 July, 2014. This conference is part of the Multi Conference on Computer 
Science and Information Systems, 15 - 19 July 2014, which had a total of 738 submissions. 

The e-Leaming 2014 conference aims to address the main issues of concern within 
e-Learning. This conference covers both technical as well as the non-technical aspects of 
e-Learning. 

The conference accepts submissions in the following seven main areas: Organisational 
Strategy and Management Issues; Technological Issues; e-Learning Curriculum 
Development Issues; Instructional Design Issues; e-Learning Delivery Issues; e-Learning 
Research Methods and Approaches; e-Skills and Information Literacy for Learning. 

The above referred main submission areas are detailed: 

Organisational Strategy and Management Issues 

- Higher and Further Education 

- Primary and Secondary Education 

- Workplace Learning 

- Vocational Training 

- Home Schooling 

- Distance Learning 

- Blended Learning 

- Change Management 

- Educational Management 

- Continuous Professional Development (CPD) for Educational and Training Staff 

- Return on e-Learning Investments (ROI) 

Technological Issues 

- Learning Management Systems (LMS) 

- Managed Learning Environments (MLEs) 

- Virtual Learning Environments (VLEs) 

- Computer-Mediated Communication (CMC) Tools 

- Social Support Software 

- Architecture of Educational Information Systems Infrastructure 

- Security and Data Protection 

- Learning Objects 

- XML Schemas and the Semantic Web 

- Web 2.0 Applications 
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e-Learning Curriculum Development Issues 

- Philosophies and Epistemologies for e-learning 

- Learning Theories and Approaches for e-learning 

- e-Learning Models 

- Conceptual Representations 

- Pedagogical Models 

- e-Learning Pedagogical Strategies 

- e-Learning Tactics 

- Developing e-Learning for Specific Subject Domains 

Instructional Design Issues 

- Designing e-Learning Settings 

- Developing e-Learning Pilots and Prototypes 

- Creating e-Learning Courses 

- Collaborative learning 

- Problem-based learning 

- Inquiry-based learning 

- Blended learning 

- Distance learning 

- Designing e-Leaming Tasks 

- E-learning activities 

- Online Groupwork 

- Experiential learning 

- Simulations and Modelling 

- Gaming and edutainment 

- Creativity and design activities 

- Exploratory programming 

e-Learning Delivery Issues 

- e-Delivery in different contexts 

- Higher and Further Education 

- Primary and Secondary Schools 

- Workplace Learning 

- Vocational Training 

- Distance Learning 

- Online Assessment 

- Innovations in e-Assessment 

- e-Moderating 

- e-Tutoring 

- e-Facilitating 

- Leadership in e-Leaming Delivery 

- Networked Information and Communication Literacy Skills 

- Participation and Motivation in e-Learning 

e-Learning Research Methods and Approaches 

- Action Research 

- Design Research 

- Course and Programme Evaluations 

- Systematic Literature Reviews 

- Historical Analysis 

- Case Studies 


xii 



- Meta-analysis of Case Studies 

- Effectiveness and Impact Studies 

- Evaluation of e-Learning Technologies 

- Evaluation of Student and Tutor Satisfaction 

- Learning and cognitive styles 

- Ethical Issues in e-learning 

e-Skills and Information Literacy for Learning 

- Teaching information literacy 

- Electronic library and information search skills 

- ICT skills education 

- in schools and colleges 

- for business, industry and the public sector 

- in adult, community, home and prison education 

- informal methods (peer groups, family) 

- Education for computer-mediated communication skills 

- Netiquette 

- Online safety for children and vulnerable users 

- Cybercrime awareness and personal prevention 

- Student production of online media 

- Web design 

- Digital storytelling 

- Web 2.0 tools 

- etc. 

- Digital media studies 


The e-Learning 2014 conference received 194 submissions from more than 35 countries. 
Each submission has been anonymously reviewed by an average of four independent 
reviewers, to ensure that accepted submissions were of a high standard. Consequently only 
33 full papers were approved which meant an acceptance rate of 17 %. A few more papers 
were accepted as short papers, reflection papers and posters. An extended version of the 
best papers will be selected for publishing as extended versions in the Interactive 
Technology and Smart Education (ITSE) journal (ISSN: 1741-5659) and also in the IADIS 
International Journal on WWW/Internet (ISSN: 1645-7641). Other outlets may also 
receive extended versions of the best papers, including journals from Inderscience. 

Besides the presentation of full papers, short papers, reflection papers and posters, the 
conference also included one keynote presentation from an internationally distinguished 
researcher. We would therefore like to express our gratitude to Neil Morris, Professor of 
Educational Technology, Innovation and Change and Director of Digital Learning, 
University of Leeds, UK, as the e-Learning 2014 keynote speaker. 

A successful conference requires the effort of many individuals. We would like to thank 
the members of the Program Committee for their hard work in reviewing and selecting 
the papers that appear in this book. We are especially grateful to the authors who 
submitted their papers to this conference and to the presenters who provided the 
substance of the meeting. We wish to thank all members of our organizing committee. 
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Last but not the least, we hope that everybody will have a good time in Lisbon, and we 
invite all participants for the next edition that will be held in Las Palmas de Gran Canaria, 
Spain. 


Miguel Baptista Nunes, University of Sheffield, United Kingdom 
Maggie McPherson, University of Leeds, United Kingdom 
e-Learning 2014 Conference Program Co-Chairs 

Piet Kommers, University of Twente, The Netherlands 

Pedro Isaias, Universidade Aberta (Portuguese Open University), Portugal 

MCCSIS 2014 General Conference Co-Chairs 

Lisbon, Portugal 
July 2014 
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KEYNOTE LECTURE 


MOVING HIGHER EDUCATION FORWARD IN THE 
DIGITAL AGE: REALISING A DIGITAL STRATEGY 


Neil Morris 

Professor of Educational Technology, Innovation and Change and Director of Digital 

Learning, 

University of Leeds, UK 


ABSTRACT 

Digital technologies have matured rapidly over the past five years, and are commonplace in 
many sectors which affect our daily lives. In the Higher Education sector, digital technologies 
are still seen as an add-on to the learning experience, and are not always an integral and 
accepted part of the teaching and learning process. Many universities have developed e-learning 
or digital strategies to address this, and this paper will reflect on the success of these strategies 
from a range of perspectives, using the University of Leeds as a case study. The session will 
explore the adoption of digital technologies by staff and students, the impact on the teaching and 
learning process and the impact on student learning and experience. The aim of the paper is to 
illustrate the benefits of a strategic approach to integrating digital technologies into a blended 
learning curriculum for staff and students. The paper will use the author’s experiences of VLE 
resources, mobile learning. Open Educational Resources, Massive Open Online Courses and 
social media to demonstrate the main arguments. 




Full Papers 




International Conference e-Leaming 2014 


CULTURE, GENDER AND TECHNOLOGY-ENHANCED 
LEARNING: FEMALE AND MALE STUDENTS' 
PERCEPTIONS ACROSS THREE CONTINENTS 
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ABSTRACT 

With the on-going “Learning Culture Survey”, we aim to foster the implementation of culture-sensitive education. The 
motivation of this study is based on the need of a better understanding of the reasons for intercultural conflicts in 
education. These issues are particularly pertinent to international learning scenarios, such as in urban education, or 
Internet-based e-Learning. The results of this research are geared towards a development of activities that prevent 
students from losing their initial learning motivation. With our standardized questionnaire, we collected and analysed 
data from Germany, Ghana, and South Korea. In such a comparative culture-related analysis, the population is usually 
considered as a whole, regardless of the respondents’ socio -cultural differences and assuming a single representative 
value per item. In this paper, we first analyse and discuss the results of our questionnaire' section “ Gender Issues”. 
Afterwards, we analyse the overall questionnaire data to focus on the extent to which female and male students’ answers 
differed. Finally, we engage in a discussion to what degree these differences impact the design of e-Learning scenarios. 


KEYWORDS 

Gender, Higher Education, E-Learning, TEL, Cross Cultural Study, Learning Culture Survey 


1. INTRODUCTION 

Increasing globalisation and mobility of learners and faculty is inevitably reflected in a cultural diversity in 
educational scenarios. On the one hand, increased cultural diversity presents itself as a very positive 
development: It helps learners to achieve competences in intercultural communication and collaboration. On 
the other hand, if the learners’ experiences are not accordingly reflected, frustration through perceived 
intercultural conflicts could emerge. Loss of motivation could be a consequence, which is directly related to 
higher dropout rates. We consider education as a process in which learners are guided on their way to 
transform experiences into knowledge. Thus, we are looking for ways to support both students and educators, 
to better understand and deal with socio-cultural diversity in education. For this paper, we investigated 
learners’ perceptions of education in different national scenarios through the lens on their gender as to 
understand its impact on e-Learning. 

The “Learning Culture Survey” (LCS) is designed as an international study. The first data collection 
phase started in 2009/10; LCS is projected to continue for at least next decade. Our research focuses on 
supporting students to improve their learning outcomes by reducing cultural conflicts in education. For this 
purpose, we investigated learners’ perceptions in different national and regional contexts. Originally, this 
investigation was designed to enhance the understanding regarding the relationship between culture and 
education. In particular, we were interested in answering questions how cultural bias affected the students’ 
perceptions and expectations towards education and how we could improve the quality of education by 
taking such diversity into account when designing learning scenarios and materials. 

Our hitherto accumulated insights led to a higher awareness regarding the character and impact of cultural 
diversity in education. We understand culture as “ the customs, beliefs, social structure, and activities of any 
group of people who share a common identification and who would label themselves as members of that 
group ” (Oetting 1993). As for practical scenarios, the results are being used to: 
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• Improve the preparative work of students and faculty members in terms of mobility; 

• Support the students’ and instructors’ development of intercultural competences; 

• Determine preventive activities to avoid cultural conflicts; 

• Design culture-sensitive learning contents; 

• Sensitise moderators of international learning groups regarding cultural conflict potential; 

• Define conflict potential for learning resources that are to be adopted to new contexts. 

The latter issue is eminent for the reuse of educational material, which is defined as one of the major 
advantages of e-Learning (Littlejohn 2003, Derntl & Motschnig-Pitrik 2003) and is particularly relevant for 
the further exploitation of Open Educational Resources (Richter & McPherson 2012; Richter 201 1). 

In the following, we use the term “Technology Enhanced Learning” (TEL) instead of “e-Learning”. In the 
context of TEL, the Internet often is used to involve learners in a collaborative learning and authoring 
activities, provide learning material through Learning Management Systems or online publishing services, 
communicate with the learners via synchronous and asynchronous channels, and provide online assessments. 
Even in regions with low population density, the relatively low technological preconditions to provide TEL 
through the Internet often are fully met by both the institutions and the learners. Thus, involving students 
from very diverse contexts in a single course generally is feasible. In such settings, particularly the students’ 
development of intercultural competences actually could be fostered. However, this option often keeps 
unused due several reasons: Prior research found that the most significant barrier for providers of TEL is the 
fear of unwillingly causing cultural (and other) conflicts (Richter & Ehlers 201 1). 

While we already were able to answer some general questions regarding the character of culture in 
education and educational culture (see section 2.2), raise our level of understanding regarding the impact of 
cultural influences on education, and determine improvement potential for several educational settings and 
scenarios, many issues still are not fully understood, and even completely new questions disclosed. 

One of these not yet fully understood issues on which we focus in this paper, is the relationship between 
gender and culturally biased perceptions of students in education. 

This paper is structured as follows: First, we provide an overview regarding the state of the art of research 
regarding “gender and TEL”. Second, we introduce the design, setting and so far achieved general results of 
our Learning Culture Survey. Third, we present the analysis of results from our questionnaire’s section 
“Gender Issues” and investigate our full data sets according to imbalances between female and male 
respondents. Finally, we discuss design implications for TEL. 


2. THEORETICAL BACKGROUND 

2.1 Gender and Technology Enhanced Learning 

Gender differences and their effects on education have been thoroughly investigated in traditional classroom 
education. Issues regarded behaviour in groups, communication styles and patterns, generally different types 
of learning motivation, and barriers towards technology usage. While studies in the early 1990s indicated 
negative attitudes of women towards computers, in early two thousand, “no significant differences between 
the genders in terms of competencies in the usage of general computer software as well as networking 
software ” remained (Atan et al., 2002, p.123). Bhushan (2008) confirmed these findings in the context of 
TEL in higher education. As major gender-specific issues in Computer Mediated Courses, Gunn et al. (2002) 
found differences in the self-reported levels of confidence, the ability to work successfully with technology, 
the use of support systems, different treatment according to laud and critique, and different patterns of 
interaction. Irani (2004) investigated differences in the self-reported levels of confidence in a longitudinal 
study. One of the findings of this study state that female learners felt frustration, particularly because of 
lacking support in technology-related tasks, such as completion of computer programming task. The study of 
Bostock and Lizhi (2005), which was focused on the use of asynchronous media, confirmed gender-related 
differences in communication patterns: Female students wrote more messages in all-female groups than in 
mixed-gender groups; while male students, in contrast, wrote more messages in mixed groups. However, the 
online discussions in all groups had a similar cognitive quality. Mikk and Luik (2005) analysed the 
perceptions of girls and boys (15-16 year old education) regarding the use of electronic textbooks and found 
that “ electronic textbooks with a high complexity of navigation and design of information endanger the 
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learning efficiency of girls” (p. 178). McSporran and Young (2001) found males at a disadvantage, as their 
skills to self-organize their learning processes and to engage in multitasking (dealing with external 
interruptions) were less developed. Yukselturk and Bulut (2009) also investigated gender differences 
according to self-regulated learning: They found “ test-anxiety ” as a significant variable for female and ^self- 
efficiency” and “ task value” as significant variables for male learners’ achievement (p.20). 

Our chosen topics for the questionnaire utilize and extend the framework proposed by Gunn et al. (2002): 
We investigate the necessity of gender-sensitive content design, the perceived confidence according gender- 
specific abilities to study social or technological issues and the perceived entry barriers, perceived treatment 
at the end of tasks, and the value of mixed gender and gender-separated workgroups. 

2.2 The Learning Culture Survey: Background, Setting, and Priory Achieved 
Research Results 

We designed and implemented a standardized questionnaire, which focuses on issues that generally are 
considered being culturally influenced. For our study, we defined the following thematic blocks: 

• relationship between learners and instructors; perceptions towards laud and admonition; group 
building processes; communication style; behaviour in groups; ( Hofstede & Hofstede 2005) 

• time management; (Hall and Hall 1980) 

• value of errors; the type of user activity; expectations towards personal coaching; (Henderson 1996) 

• demand to influence learning contents; (Trompenaars and Hampden-Turner 2006) 

• how and when feedback is to be provided. (Noelting et al. 2004) 

• gender issues (Gunn et al. 2002) 

As for the operationalization of these topics, we focused on issues that reportedly caused conflicts in 
educational scenarios. We eventually defined a total of 100 culture-related items, which we asked the 
students to evaluate on a four-point Likert scale (fully agree - fully disagree). We implemented a “force- 
choice” design (Lenski & Leggit 1960) because we wanted the participants to take a position that gets closest 
to what they actually think instead of expressing that they are undecided. With that decision we risked 
unwanted distortions (Garland 1991). Thus, we offered the opportunity to express if an item generally is not 
applicable to a particular context. This fifth option was positioned apart of the scale and, so far, has rarely 
been used. The questionnaire was pre-tested and was modified accordingly. 

The first part of our investigation included Germany and South Korea, which according to the ATLAS 
survey from Muller et al. (2000) are the only countries that were considered being culturally homogenous. 
The questionnaire was provided in each of the national languages. In Germany, we designed the survey as an 
in-depth study and used the online version. We had the chance to take full samples (inviting all students) of 
three regionally distributed universities (1,817 completed questionnaires, 2-5 % acceptance rate). To 
determine the scope of our results we needed a sample where we could distinguish between the results of the 
various faculties. As for South Korea, we chose a broad-based design by drawing from 39 universities from 
which we obtained 286 completed questionnaires. In this study, we wanted to explore if the responses show 
significant differences amongst the universities. Due legal reasons, we had to conduct the questionnaire in its 
paper-based version. In order to achieve at least a quasi-random sample, we selected the students by 
following a random route algorithm (Kromrey 2006). The acceptance rate was around 50 %. For the analysis 
of the responses, we followed the recommendation of Baur (2008) for ordinal-scaled data: We binarised the 
results into positive and negative outcomes and focused our analysis on the percentage of positive answers. 

One of the most relevant questions was related to the contextual transferability of our results. Hofstede & 
Hofstede (2005) suggest that culture, if related to value -systems, generally is a matter of national bias, and 
without regard of the context in which culture-specific results were achieved, these are transferable to any 
other context. This assumption, however, was challenged. Jandt (2004) for example expresses that “ cultures 
do not respect political boundaries. Border cities such as Juarez, El Paso, Tijuana, and San Diego can 
develop cultures that in some ways are not like Mexico or the United States” (p.7). Inglehart and Wetzel 
(2010, p.555) argue that “ National means tell only part of the story. Measures of variance and skew within 
societies are also informative” . 

Figure 1 illustrates three general results according to the questionnaire’s topic “Role of the Lecturer”, 
very similar answer patters between faculties (1), significant national differences and the value of the 
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spectrum of answers (2), and fully explainable differences between the context of higher education and 
professional training in Germany (3). We contrasted the results of our in-depth study in Germany according 
to the different faculties within each university and the average results of the three universities. We found a 
certain level of variability between the answers of the different faculties (including outliers) but after 
visualizing the results in a net diagram, we found that all responses followed a certain pattern. In the upper 
left of Figure 1/1, the results from the different faculties of the University of Cologne are exemplarily 
displayed. The same result was found between the average values of the German universities. 




Figure 1 . Role of the Lecturer: Learning culture in different settings 

The answers that were received from students in the different universities in the South Korean sample 
also showed a particular answer-spectrum regarding most of the items. Yet each pattern again, was quite 
similar (apart of one university that just provided extra-occupational master programs. The patterns of the 
German and the Korean universities, however, were completely different from each other. In the upper right 
section of Figure 1/2, the answer-patterns from both, the German and the South Korean survey are displayed 
including each answer spectrum. What we can recognize in this figure is particularly that there are sections in 
each of the national contexts, which exceed the areas of the other. We came to the conclusion that the 
average percentage value is suitable to understand that generally there are differences but not sufficient to 
provide explanations for conflicts or emerging adaptation needs. Instead, we found that the spectrum of the 
answers within one context is a more meaningful indicator. Provided that local students are expected to cope 
with the educational style of any university within the country, and assuming that cultural bias at least to 
some extent is related to personal experiences (reflected in the answers), the answer-spectrum of all 
universities within a country could be understood as an indicator for the level of acceptance (Pless & Maak 
2004) regarding diversity in education. If this is the case, then conflicts in intercultural education might 
particularly occur when a student from the one context meets conditions that are outside of the spectrum of 
his/her own context. In the figure, we marked related areas that accordingly indicate a high risk for conflicts 
with red and black arrows. 

In the next step, we wanted to find out if higher education and professional training reveal similar results 
(Figure 1/3, centre). From the 30 invited German stock-noted enterprises, five granted their support and 
involved a small number of employees (each 25). From two of these enterprises, we eventually received 
seven and more completed questionnaires (pencil-and-paper form). Even though such small samples are far 
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from representative, we found surprising results: The responses of the two enterprises were quite different 
based on the results across the universities. All found differences could be explained considering specific 
characteristic of each enterprise/context. We concluded that generalizing our results from higher education to 
other educational contexts is inappropriate (Richter & Adelsberger 2012). 

Previous literature suggests that cultural bias does not yet have its full impact on children below an age of 
12 years but instead their curiosity has a higher influence on decision taking. In this context, Mitra et al. 
(2005) reported from their research project, which was highly related to curiosity, that children above 12 
years did not like to participate or at least, quickly lost their interest. Buehler et al. (2012) investigated 
culture-related perceptions towards the experimental use of unknown technology. They found that children 
above the age of twelve years compared to younger children, reacted with the (for their particular cultural 
contexts) expected wariness. Thus, we did not investigate the context of school education. 

Even with the limitation to a single educational context, another issue emerged according to country-wide 
transferability: How about countries that can not be considered culturally homogenous, such as countries in 
which several societies were joined into a single nation during the times of colonialisation, or in which more 
than one national language was spoken (Condon & Yousef 1975)? 

By chance, we conducted a paper-based test study (same questionnaire, translated to English and French) 
in two universities in Cameroon, one located in the English and the other in the French language region; we 
received 30 completed questionnaires from each. We conducted an a-priori analysis considering different 
thematic blocks of the questionnaire: With a chance of over 98 %, each student’s dataset was correctly 
appointed to the one or the other university. This is a strong indicator that transferring locally achieved 
results to the whole population of a country that culturally is not homogeneous is questionable. If a 
comprehensive understanding of culture shall be achieved, at least, regionally more distinguished 
investigations are required. 


3. STUDY DESIGN 

For this paper, we use our results from the German and the South Korean context and focus on the thematic 
section “Gender Issues”. In 2013, we collected data from another national context, which was Ghana. This 
sample included respondents from the University of Ghana in Accra. We conducted this survey in its English 
language online version. The general conditions and design were the same as for the German sample and as 
described in section 2.2. For the Ghana study, a separate questionnaire-instance was set-up and the related 
link to the questionnaire was included in the letter of invitation. 

In contrast to Germany and South Korea, Ghana is not a culturally-homogenous country. More than 100 
different ethnic groups were united into a common national context during the periods of colonialisation. 
Thus, the results neither allow to draw conclusions regarding the specific societies’ cultures, nor are they 
representative for the “general culture” in the country (whatever this might be) but just reflect this particular 
university’s population. Due to the following reasons, we think that the results still provide an insight 
regarding the country’s culture in higher education : First, students in Ghana enter the universities with more 
or less common experiences from their earlier education: Ghana has nine years of compulsory education, a 
national educational system, and a national curriculum. Teaching usually takes place in the official national 
language. Second, the number of public universities without particular thematic specializations is quite 
limited (6 public universities in total, 3 without specialization). The collected demographic data showed that 
students from all over the country frequently attend the University of Ghana in its capital Accra. For a 
specific analysis regarding particular regions, the number of responses per region was too small (306, 
acceptance rate 1,54 %). 


4. RESULTS 

In Table 1, in the first column, the original statements of the question block “Gender Issues” are displayed as 
they were to be evaluated on the 4-point Likert scale. On the right side are three blocks, each with three 
columns that display the percentage of positive female, male and average responses from the countries 
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Germany, Ghana, and South Korea. According to an imbalance between female and male answers, we define 
a deviation below 5 % (absolute) between male and female responses as standard error. 


Table 1 . Gender Issues: results of universities from 3 continents (percentage of positive answers) 



Germany 

Ghana 

South Korea 

female 

n=544 

male 

n=1268 

average 

female 

n=126 

male 

n=177 

average 

female 

n=153 

male 

n=131 

average 

a. Learning content should be designed 
in the same way irrespective of the 
learner’s gender. 

94.12 

92.91 

93.29 

86.51 

90.40 

88,89 

76.47 

75.57 

75.79 

b. Women and men have the same 
ability in understanding complex 
technical information (e. g. in the 
domains of engineering, mechatronics). 

72.61 

77.07 

75.78 

66.67 

70.62 

68,95 

45.10 

52.67 

48.42 

c. Women and men have the same 
ability in understanding social domains 
(e. g. in the domains of pedagogic, 
gerontology or primary education). 

70.77 

80.69 

77.77 

80.95 

77.40 

78,76 

59.48 

54.96 

57.19 

d. Women and men are treated the same 
way when completing a task 
successfully. 

56.99 

54.06 

54.93 

66.67 

74.58 

71,24 

40.52 

48.85 

44.56 

e. Women and men are treated the same 
way when failing a task. 

47.24 

47.75 

47.61 

58.73 

66.67 

63,40 

35.29 

47.33 

40.70 

f. Women and men have the same 
chances to access studies on all 
subjects. 

75.74 

75.97 

75.89 

79.37 

82.49 

81,37 

64.05 

72.52 

68.07 

g. Workgroups intellectually benefit if 
the genders are represented in a well- 
balanced ratio. 

75.92 

84.16 

81.67 

84.13 

81.92 

82,68 

80.39 

75.57 

78.25 

h. A separation of gender in the learning 
process eases the social interaction 
within groups. 

10.66 

7.2 

8.64 

46.83 

42.94 

44,44 

31.37 

27.48 

29.47 

i. It generally is not useful to implement 
a quota for the number of women in 
supposedly men dominated areas. 

27.76 

46.26 

40.67 

52.38 

61.58 

57,84 

40.52 

38.93 

40.00 


Figure 2 provides the first visualization of the gender-specific differences between the responses 
according to the results from the thematic section “Gender Issues”. We used separate bar diagrams for each 
country. We found 10 cases amongst the three countries, in which the answers revealed to be significantly 
imbalanced in the gender-specific contrasting. Across the countries, however, there was no clear-cut pattern 
that would suggest imbalances regarding particular items. A similar result could be found throughout the 
whole datasets: 

In the full South Korean sample (100 items), 27 items showed an imbalance above 5 % between the 
responses of the female (f) and the male (m) students; regarding 2 items, the level of imbalance exceeded 10 

%: 

1. Gender Issues: “ Women and men are treated the same way when failing a task.” (35.29f : 47.33m) 

2. Motivation: “ lam easily discouraged because of others or situations.” (48.37f : 38.17m) 

In the full German sample (100 items), we found 30 items in which the imbalance between the answers of 
the female and the male students exceeded 5 % and amongst those, the responses in 5 cases exceeded 10 %: 

1. Feedback: “ For me, it is ok when critical feedback in the learning process is given in front of my 
colleagues.” (77.39f : 63.99m) 

2. Motivation: “ lam easily discouraged because of others or situations.” (42.28f : 53.1 lm) 

3. Motivation: “ I experience being motivated if the imparted knowledge is strongly needed for upcoming 
examinations, tests, and/or presentations.” (56.52f : 66.67m) 

4. Gender Issues: “It generally is not useful to implement a quota for the number of women in supposedly 
men dominated areas.” (27.76f : 46.26m) 

5. Group Work, evaluate statements: “It should be possible to divide a given task into subtasks with 


8 



International Conference e-Learning 2014 


similar complexity for being solved solely by each participant within the group.” (58.64f : 71.47m) 

In the full sample of the university of Ghana, we found 29 cases where the answers from female and male 
students diverged accordingly (over 5 %) and in 4 cases, the 10 % were exceeded: 

1. Motivation: “ lam easily discouraged because of others or situations .” (26.98f : 37.29m) 

2. Group building process: “/ try not to actively exert influence on the arrangement of the group 
members but wait until lam invited to participate in a group.” (51.59f : 39.55m) 

3. Group Work, evaluate statements: ”When working in a group, I feel confident in presenting my own 
opinion to the other group members.” (95.24f : 84.75m) 

4. Role of the Teaching Assistant: “In my opinion a teaching assistant occupies the role of a respect 
person” (26.98f :41.24m) 


Gender Issues; 

Korea HE - Female vs. Male 

0,00 20,00 40,00 60,00 80,00 100,00 


Contents should be the same 
Same abilities in tech. Issues 
Same abilities in social issues 
f&m arc treated same if successful 

f&m arc treated same if failied "Ko fern pos (n-153) 

Ko male pos (n=131) 

f&m equal chances to study anything 
workgr. intem.prof. from. cqu. gender distrib. 
f&m separated in workgr. cases social 
f-quota not useful in m.-domin. Field 
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Germany HE - Female vs. Male 
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Gender Issues; 

Ghana HE - Female vs. Male 
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Gh male pos (n=177) 


Figure 2. Gender Issues: Contrasting results from female and male students per item 


There was a single item in which the responses of the female and the male students from all thee countries 
showed a similar high imbalance, which was “I am easily discouraged because of others or situations ” in the 
thematic block “Motivation”. While the level of imbalance was similar, the answers generally were not. In 
the South Korean sample, less female students felt the risk of being discouraged than male students. The 
samples from Germany and Ghana revealed the opposite. We have no explanation for this phenomenon; also 
Hofstede’s Masculinity Index cannot provide an explanation, since in this dimension, Germany (66/100) and 
South Korea (59/100) are quite close to each other and Ghana is not included. 

We display the results of the thematic block “Gender Issues” in a net diagram in Figure 3. It shows that 
between the three countries, there generally are cultural differences regarding most considered items. An 
exception is the item “ Workgroups intellectually benefit if the genders are represented in a well-balanced 
ratio ” where the distance between all three national samples is below 5 %. The largest cultural distance can 
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be found between Germany and South Korea (27.36 %) regarding the item “ Women and men have the same 
ability in understanding complex technical information (e. g. in the domains of engineering, mechatronics)” , 
directly followed by the item “ Women and men are treated the same way when completing a task 
successfully ” (26.68 % difference between Ghana and South Korea). 

Gender Issues (HE); 

Korea, Germany, & Ghana: female vs. male students 


Contents should be the 
same 
100, OOt 



Ghana HE Average (%p; n=306) 

Gh fem pos (n=126) 

Gh male pos (n=177) 


Figure 3. Gender Issues: Students’ responses from South Korea, Germany, and Ghana contrasted in a net diagram 

Regarding the treatment in cases of failed and succeeded tasks, a major imbalance between the female 
and the male students’ responses from Ghana and South Korea were found. In both countries, the male 
students show a significantly higher confidence that both genders are equally treated than the female 
students. As for Ghana, this seems not to be a larger issue because both responses are in the positive interval 
of the scale (above 65 % positive). In Korea, in contrast, almost 65 % of the female students report an 
imbalance in treatment (35.29 % positive answers). In accordance of the almost 50 % of positive answers 
regarding easy discouragement (Motivation), this result is somewhat alarming. The corresponding result from 
the German sample also is not fully reassuring: Particularly regarding the treatment in case of failures, less 
than 50 % of all students expressed a balanced treatment according to the gender (but their assessment is 
similar). 

According to the perceived confidence of female and male students in relation to gender-specific abilities 
to study in particular fields, the results were consistent across all three countries. In contrast to our 
expectation that male students (particularly in rather masculine societies) would generally express 
technological issues as their particular domain, more male students expressed that gender has no significant 
influence on the ability to study technical issues. Female students concordantly expressed a higher 
confidence that both genders can cope with study fields that are related to social issues. 


10 


International Conference e-Leaming 2014 


5. LIMITATIONS OF THE STUDY 

The general limitations of our survey have already been introduced in the sections 2.2 and 3. In this study, we 
applied an analysis in which we distinguished the results according to the gender of the respondents. We did 
not take into consideration the extent to which gender of the instructor might play a role for the students’ 
perceptions of education. This could particularly be relevant when it comes to questions like the role and 
tasks of the instructor and shall be investigated in future (but separate) studies. 

We did not try to explain the differences found amongst the specific cultures. Such explanations require a 
very profound understanding of each culture, and should keep reserved for local people. For our purposes 
(recognising conflict potential) it is sufficient to identify such differences. 


6. CONCLUSION AND IMPLICATIONS FOR TEL 

What our investigation generally showed is that analysing cultural phenomena can help recognizing patterns 
on how a particular society functions. However, such patterns do not reveal a complete story: Even if results 
can be considered representative for a society, they still do not reflect particular gender differences according 
to the investigated perceptions. In the context of higher education we found that even if following the same 
or very similar cultural patterns, such differences in perceptions of male and female students can actually be 
substantial. When aiming to reduce conflicts in educational scenarios and support students to keep their 
motivation on the highest possible level, such gender-related differences in perceptions of education can turn 
out to be significant and thus, need to be considered in the educational design. All reported findings of our 
study are relevant for both traditional education and TEL. Thus, implications and recommendations as 
provided in this paper can serve as recommendations for education in all settings. 

Gender -related fairness regarding treatment after completed tasks in general and particularly after failure 
needs to be addressed in the German context. For Germany this is a major issue because what appears to be 
commonly recognized by both female and male students, violates the law. In the German context, no further 
peculiarities were found. All other items are located in the positive area, mixed gender workgroups are 
preferred, and the implementation of a female quota is somewhat undecided (almost normal distribution). 

In the sample from Ghana, gender-related issues were all answered on a positive level from both female 
and male students. A higher divergence between the genders was found regarding the treatment after failing a 
task but both groups expressed a positive impression of fair treatment. In contrast to Germany and South 
Korea, students seemed to recognize a certain value in gender-separated education and workgroups. 

In the context of South Korea, several items are located in the area between 40 % and 60 %, which 
indicates a rather individual than culturally biased evaluation. Equal treatment after finalized tasks is also an 
issue. Korean students do not prefer gender-separated groups but would rather have a female quota when it 
comes to the accessibility to male-dominant fields of study. 


7. FURTHER STEPS 

The Learning Culture Survey is driven forward by chance and is highly dependent on (mainly) voluntary 
support through universities. Since the questionnaire is to be conducted in the local languages, the 
availability of translated versions is a crucial precondition for investigations. Currently, questionnaire 
versions are available in Bulgarian, English, French, German, Greek, Japanese, Korean, Portuguese, Russian, 
and Turkish. All language versions are (being) implemented in our online survey system. 

Due to data protection regulations, we cannot directly address the students in foreign universities. Thus, 
in order to proceed, we would like to invite universities from all over the world to support the Learning 
Culture Survey, be it through sending invitations for participation to their local students (after making an 
arrangement with us) or through contributing further translations. Once, the data collection in a particular 
university is completed, we are willing to share the results with the supporting university. 

Our long-term objective is to provide our data in a publicly available database. 
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ABSTRACT 

Research in learning technologies has often focused on the affordances of single technologies such as pcs, smartphones 
or interactive whiteboards. However, in most learning environments technologies do not stand alone but are embedded in 
activity structures and webs of materials that make up the learning activity. This is specifically relevant when the object 
of study is mobile learning, where devices are flexible and follow the learner in his/her shifting learning activities and 
needs. This paper focuses on the ways in which iPads as learning technologies become involved in sociomaterial 
practices that are emergent and improvisational and how this contributes to educational change. The paper proposes that 
the introduction of tablets into classrooms will enroll devices in networks of learning that establish new and significant 
relationships between learning technologies such as iPads, whiteboards, pcs as well as books, pens and paper. These 
relationships challenge the idea that iPads can act as isolated and unique actors in educational development. The paper 
focuses on the ways in which teachers negotiate material cultures in schooling when iPads are involved in learning, i.e. 
how they become central actors in linking, translating and creating trajectories between the artefacts of learning. I study 
these processes from three perspectives that I identify as respectively capturing, bricolaging and building knowledge. 


KEYWORDS 

Mobile learning, material learning, Actor-Network Theory, teachers’ professional competences 


1. INTRODUCTION 

John is a biology teacher in a school in the west of Denmark where teachers and pupils have recently been 
given iPads to use in the school year 2012-13. John likes to use both the interactive whiteboard and the 
blackboard to illustrate and explain processes such as the photosynthesis to pupils. He uses PowerPoints, 
illustrations from the internet, his own word files and chalk to work on the two boards at one end of the 
classroom. John also requires pupils to take notes on their iPads while he is talking, and sometimes to look in 
their books and check factual information while he is explaining the processes of photosynthesis. 

John is one of several teachers in the teacher community at Middletown school in Denmark who organize 
their teaching by associating learning activities with different kinds of artefacts such as blackboards, 
whiteboards, pens, books, paper and iPads. The iPads, a promising new technology recently invested in by 
the school, are activated within this set of complex material and organizational practices that make up the 
learning of biology. In this web of learning materials, every artifact contributes to shaping the learning 
experience by adding to and acting in the chain of associations that John competently creates while he is 
teaching. 

John’s teaching is an example of practice that illustrates how learning technologies gain their significance 
by participating in a web of relationships which are materially and socially enacted as a part of schooling. As 
an example, John’s teaching contradicts the idea that technologies such as the iPad are isolated agents of 
change in the culture and practice of schooling. In fact, such an idea will, as formulated by John Law, 
“conceal for a time the process of translation itself and so turn the network from a heterogeneous set of bits 
and pieces each with its own inclinations, into something that passes as a punctualised actor” (Law 1992, p 
386, in Fenwick and Edwards 2012). 

This paper aims to identify and follow the ‘heterogeneous bits and pieces’ orchestrated in teaching and 
how they make sense through links and translations. The paper builds on data from a research project that 
investigated the role of iPads in the learning of five classes of seventh graders (age 13-14) of which two were 
special needs classes. I followed these classes of seventh graders for 3 months of the school year, doing 
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observations and interviews with teachers and pupils. The paper builds on the argument that schooling is a 
material culture in which artefacts are embedded in different kinds of activities that make up the learning and 
that iPads make specific contributions to these material cultures of learning. 


2. SCHOOLING AS A MATERIAL CULTURE 

Materialities and their significance in cultures of education are rich and relevant points of departure for 
studies in educational technologies as resources and their social meanings, qualities and agencies are central 
to the ways in which teaching and learning can be envisioned and performed (Waltz 2006, Nespor 2002). In 
fact, without the aid of materials for e.g. writing, drawing, listening, calculating and laboratory work teaching 
can neither, it seems, be envisioned nor enacted (McGregor 2003, Meyer 2013). Highly profiled technologies 
such as the iPad is just a recent addition to this complex web of artefacts in schooling which shapes the ways 
in which learners can learn and teachers can teach. 

Though studies in the materiality of schooling can provide rich empirical knowledge about the ways in 
which artefacts are embedded in learning and learning environments, material objects and their agencies 
have, according to a number of recent studies largely been disregarded. Most prominent among these studies 
is research inspired by Actor-Network Theory (ANT) (Latour 2005, S0 re risen 2009, Johri 2011, Fenwick, 
Edwards and Sawchuk 2011, Fenwick and Edwards 2012). These studies in various ways argue for the 
legacy of ANT in foregrounding the role of material cultures in learning. 

According to Bruno Latour one of the significant contributions of ANT has been “to have transformed the 
social from what was a surface, a territory, a province of reality, into a circulation” (1999, 17). In education 
studies this can be translated into an understanding of how artefacts are enrolled in webs of sociomaterial 
negotiations that constitute processes of learning such as for instance writing, reading or doing biology (cf. 
the example above). In this understanding, according to for instance McGregor (2004) artefacts should not be 
understood as passive presences or mechanistic manipulators, but as actively participating in constituting 
spaces, trajectories and relationships in education. The legacy of ANT in education is therefore to provide 
both thick and multi -directional descriptions of networked chains of activities in schooling as well as a 
renewed understanding of the ways in which artefacts participate in these networks. 

As a separate but related approach to the study of material cultures in education, historians of education 
have studied the ways in which objects have figured in and configured educational practices from the 
beginning of schooling until the present day (Lawn & Grosvenor 2001, 2005, Burke et al 2010). Lawn and 
Grosvenor (2001, 2005) for instance argue, through historical approaches to the materialities of schooling, 
that relationships between teachers and material technologies have existed since the formation of schools as 
specialist institutions in the early 19 th century. Schools, they argue, usually come equipped with - or develop 
- material cultures in which teachers have had to constitute learning by linking activities to different kinds of 
artefacts. These include basic tools such as pencils, textbooks, rulers and blackboards, but have recently 
incorporated a variety of technologies such as recorders, computers and mobile devices. These material 
cultures - established through centuries of material practices - are essentially sedimented material cultures 
that teachers and learners can draw on when engaging in a variety of educational practices. As material 
cultures, artefacts in schooling are often enrolled in “a system of the teachers’ own devising” (2001, 122) 
where teachers design or improvise with the artefacts available to them in classrooms. In this way teachers 
become central actors in linking, translating and creating trajectories between the artefacts of learning, i.e in 
negotiating the educational significance of material cultures in schooling. 

In this paper I want to look specifically at three ways in which IPads participate in teacher devised 
systems of related technologies. I identify these systems as respectively capturing, bricolaging and building 
knowledge through the linking of artefacts in networks of learning activities. Each of these systems provide 
different negotiations of iPad participation in learning - and therefore of different teachers’ associations of 
iPads with learning activities. 
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3. MATERIAL CULTURES IN MIDDLETOWN SCHOOL 

Looking at schooling through the lens of material cultures can address the ways in which resources are 
acquired, circulated, preserved or discarded in educational cultures where educational change is associated 
with the investment in technologies (Lawn & Grosvenor 2001). IPads are, as highly profiled digital devices, 
intimately associated with the ways in which schools can invest in resources, implement them in practice and 
organise them as part of the teaching. 

In Middletown school the issue of resources and their significance for learning has recently affected the 
practice of teaching in a number of ways. As is often the case with municipally funded schools in Denmark, 
Middletown school had for some time had no significant financial resources of its own. Over the years, the 
school has made some investments in laptops and interactive whiteboards, however, these were mostly made 
to support specific learners or to enhance classroom management through whiteboard technologies. When 
resources are scarce, teachers have to be resourceful and manage with whatever is available in school 
repositories, they are forced to preserve and engage with existing materials within the everyday practices of 
teaching. This is highly observable in the activities of teachers in Middletown school, cf. the example of John 
above. 

The introduction of iPads into Middletown school in the summer of 2012 brought a new generation of 
devices into the school that in a very short time enabled teachers and pupils to take advantage of the learning 
potentials promised by mobile learning devices. As an educational resource the iPad from the beginning 
stood out from the materials and educational technologies teachers had been used to working with in the 
classroom for many years, i.e mostly low-cost, dependable and reusable resources such as books, jotters and 
pens. 

As an educational resource, the iPad promised to link Middletown school with the educational innovation 
and school development associated with mobile devices - a link that, according to the school leader, might 
over time improve the profile and financial situation of the school. However, the leap into a potentially 
flexible, personalized and learner oriented educational environment was not generally followed by sufficient 
professional development of the teachers, as they were only given a couple of introductory courses to the use 
of apps which did not really relate to their teaching practices. As a result, teachers were generally forced to 
cope with the new technology on their own in the time allocated for the preparation of lessons, which did not 
leave much incentive or potential for changing the everyday practices of teaching. Therefore, the systems of 
practice that had already been established with regards to linking learning processes to a variety of 
dependable and available resources was largely maintained, and the iPad was enrolled as an additional 
educational resource in these practices. 

As an initiative to modernize schooling the incorporation of iPads into Middletown school may therefore 
be interpreted as a relative failure. However, as I shall argue, the circulations of materials in Middletown 
school nevertheless became potential systems of change, as teachers turned out to be highly competent in 
orchestrating, organizing and enacting learning processes that involved elaborate systems of activities in 
which the iPad could participate. 

In the following I shall give examples of and discuss the ways in which the iPad acted as a significant 
element in teaching in Middletown school. 


4. CAPTURING CONTENT THROUGH THE IPAD 

According to Schon (1983) professionals such as teachers constitute practices through the constant 
experimentation with and reflection in and through practice. In this way professionals become reflective 
practitioners who continually build on and expand their theories of practice. In Middletown school, teachers 
connect with both established ways of engaging artefacts in learning and transform these practices by 
incorporating artefacts in new ways through improvisation and reflection in practice. 

As mentioned above, pencils, books and blackboards constitute the basic tools of schooling, especially in 
literacy practices which are intimately related to these artefacts. In a historical perspective, “The pencil, like 
the slate pencil, was engaged with ‘imitation’ of the teachers’ work on the blackboard but it then allowed the 
use of copybooks to be developed and integrated with blackboard activities. Combining the blackboard, the 
pencil, and the copybook into an effective method of teaching, especially of the teaching of writing, was a 
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‘device’ or system of related technologies which has continued in modified form since” (Lawn & Grosvenor 
2005, 11). Learning through literacy practices can therefore be understood as a system that involves several 
artefacts in copying from one ‘device’ onto the other. 

As most of his colleagues, John, the biology teacher, has a practice based theory of learning that works as 
an often tacit basis for his orchestration of sociomaterial enactments in teaching. As most of his colleagues, 
John requires pupils to learn as a result of translating knowledge from authoritative learning resources such 
as books or PowerPoints onto their own paper or iPad. This can be observed both in John’s classes and in 
those of his colleagues, where materials abound, and where teachers orchestrate relationships between 
artefacts and activities in the process of teaching. These orchestrations define trajectories in the sociomaterial 
enactments of learning, as when John asks pupils to copy information from the blackboard or whiteboard 
onto their iPads. 

According to John the newly required iPads are user-friendly, easily accessible devices that generally 
support pupils’ learning in biology and other subjects. However, the iPads, he feels, also contribute 
negatively to pupils’ learning by eliminating or shortcutting trajectories of learning that were available to 
pupils before the shift to the tablets. For instance the pupils have discovered that they can take photos with 
their iPads when John explains something on the blackboard or whiteboard. John, however, feels that the 
activity of saving information through photographing eliminates a significant process in learning, i.e the 
process of copying down information through hand -activated note taking. In this sense, according to John, 
the addition of the iPad to the chain of material practices represents an inferior translation of subject 
knowledge and therefore constitutes inferior learning. As a consequence, John does not allow pupils to take 
photos with the iPad in class, he requires them to take notes by hand. 

Like John, his colleagues at Middletown school have discovered that pupils take pictures of and save 
notes with their iPads when they are in class. However, unlike John, the majority of his colleagues feel that it 
is an advantage for most pupils to photograph and save information on the iPad instead of copying by hand 
from the blackboard or whiteboard. Photographing, as a translation from the blackboard to the iPad, will 
enable pupils to listen more attentively to what the teacher is saying, these teachers argue, and in addition to 
this, pupils who are slow readers and writers, will not slow down the teaching when photographing replaces 
hand-activated note taking. These examples illustrate how different translations of iPad and blackboard 
relationships constitute learning in different ways, according to teachers’ theories of practice. These theories 
of practice contribute to defining the ways in which iPads can be added to the circulations of materials in 
Middletown school. 

In digitalized cultures where mobile devices are increasingly embedded in everyday practices, media 
convergence is, according to Kress (2010), a central and ubiquitous phenomenon. Smartphones and other 
portable devices for instance bring together the functionalities of formerly separate digital devices, such as 
cameras and telephones. One of the consequences of this convergence is that the capturing of images and 
copying of semiotic material from the internet has gained priority over producing written content. Kress 
argues that this process of convergence “changes the way we conceive of representational means and 
meaning-making in the world, favouring selection, ‘capture’ and transformation rather that ‘production from 
scratch’” (2010, 188). This capturing of content adequately describes the processes associated with 
translating and saving material onto the iPad in John’s class, and the ways in which pupils transform note- 
taking from a production from scratch activity to an image that can be saved. Capturing therefore becomes a 
significant activity that the iPads add to the system of sociomaterial enactments in Middletown school, an 
activity that in different ways feeds into the visual culture in the school. 


5. IPADS IN VISUAL CULTURES; CRAFTING AND BRICOLAGE 

As argued by Kalthoff and Roehl (2011) classroom teaching makes extensive use of the sense of sight. In 
teaching, knowledge is for instance often made visible by teachers’ blackboard illustrations or by objects, 
maps or diagrams designed to represent knowledge areas such as geography, science or biology. In 
classrooms, visual cultures are both part of teachers’ use of educational tools and of learners’ end products. 

John’s colleague Lucy is a teacher who often uses visual representations such as posters in her teaching. 
Posters, unlike capturing, are creative, often pupil-produced visual representations made from scratch to 
illustrate and collect aspects of a specific theme or issue. 


16 



International Conference e-Learning 2014 


Posters are visual representations of the ways in which learning is constituted through sociomaterial 
processes. Posters involve the association of both simple objects such as pens, paper, cardboard and 
technologies such as iPads. As assemblages of materialities and modes, posters both represent and involve a 
number of translations between materials, modes and activities that make up a topic or a field of knowledge. 
As a representation the poster is a “socio-material bricolage” (Jolrri 2011), i.e. it is produced as the result of a 
number of processes in which translations are made between different kinds of materials that act as mediators 
for learning. In Middletown school, posters are deeply embedded in the teaching and learning processes, 
partly as an aspect of project pedagogy, which usually involves some kind of crafting where pupils are 
required to work on their own in finding knowledge about a specific topic. 

An example of Lucy’s teaching is a project based learning process in geography where pupils have to find 
information about a country of their own choice and illustrate their knowledge by making a poster. Lucy 
presents the assignment to the pupils by giving them a colourful poster she has copied from a book she uses 
in school. The poster both visualizes topic areas such as population, nature, economy, and environment and 
in simple textboxes suggests relevant questions the pupils can work with. The poster presents itself as a 
teacher directed illustration of how visual and written representations can be assembled to make up a country 
description. 



Figure 1 . Working on the poster 


In class I follow a couple of the groups of pupils and their work on the geography project. One group is 
working on Iceland as a country, describing aspects of for instance nature and animals. The group consists of 
a girl and two boys, the boys work together on making a flag and finding factual information about Iceland. 
The girl collects and copies material onto their joint poster. In order to produce the poster, the pupils use a 
number of resources, for instance paper, cardboard, pencils and iPads. 

Making the poster involves the iPad in different ways. In the chain of materialities that are enrolled in the 
making of the poster, the iPad acts as a repository of different kinds of information that can be accessed 
through the internet. The girl for instance uses the iPad to access Wikipedia and other webpages where she 
can find information about natural phenomena and animals in Iceland. This information is translated, i.e. 
written, onto the poster by pencil and hand. One of the boys in the group is making a cardboard 
representation of the Icelandic flag. He uses Google to find pictures of the flag that he translates onto 
coloured cardboard. This translation creates links between the screen image and the cardboard which are 
different representations of the Icelandic flag as a materiality that ‘represents’ Iceland. In both cases, the 
pupils create their (visual and written) productions from scratch, i.e. they do not shortcut the learning process 
by capturing. The process of making the poster becomes a “socio -material bricolage” (Johri 2011), in which 
several modalities and materialities are activated. 
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In Lucy’s system of educational tools assembling and bricolaging itself becomes a potential end product 
of learning as is illustrated in the making of the posters. Bricolaging becomes the system that holds teaching 
and learning together as networks of sociomaterial enactments. Bricolaging also becomes the practice that 
enrolls iPads in learning as part of the circulations of materials in the classroom. 


6. BUILDING AND CRAFT CULTURES IN MATH AND HISTORY 

In Middletown School low-key technologies co-exist with cutting-edge technologies such as the iPad and 
interactive whiteboards. The process of building new knowledge therefore often involves an association of 
these different resources and incorporates their potential ‘affordances’. Building also becomes a metaphor for 
learning itself, as will be illustrated by the following example. 

In project work learning processes are often linked to artefacts that have specific qualities for learning or 
practicing particular skills. In Middletown school the history and math teacher have collaborated on planning 
a project that integrates a number of material resources that will help learners to understand mathematical and 
historical aspects of buildings and architecture. The history teacher tells me about the project. In class, he 
says, the pupils will be working with papercraft, i.e. cardboard, to understand scaling as a mathematical 
aspect of buildings. Before working in class the pupils have visited a square in the local city that they have 
measured and are reproducing in different scales in class. In class they have learned how they can work with 
cardboard by gluing the sides of cardboard together to make three dimensional buildings. In order to work 
with this skill they have cut out and built a church in cardboard. On top of the square they built from their 
visit to the city, pupils are constructing a number of different buildings that fit the scale of the square. Some 
of these buildings have a historical significance, for instance Big Ben and Brandenburger Tor. These 
buildings are built in Lego, the plastic toy brick made in Denmark. 

Onto this heterogeneous chain of materials the iPad is added as a new element that can further qualify the 
process of conceptualizing and visualizing the processes of building and scaling and the historical 
significance of buildings. The history teacher tells me how at the end of the chain, pupils are asked to build in 
Minecraft, using the original idea of the square as a basis on which to build. What building in the Minecraft 
app can add to this process is enabling the pupils to see the building from the inside, i.e. in a 3D perspective. 
Through Minecraft, pupils can build architecture, for instance tunnels, that they can look at and understand 
from the inside. 



Figure 2. Building in Minecraft 
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The chain of learning through building is - as a system - similarly to the processes described above - a 
multi-purpose chain of learning that links not only heterogenous materials such as cardboard, glue, scissors, 
rulers, software etc., but also different subject areas, skills and aspects of buildings as a historical and 
material phenomenon. In addition to this the chain of learning links local buildings with buildings in other 
parts of the world, combining the different aspects of their historical and functional significance in the study 
of ‘building’ as a learning activity. The complexity of this process of linking, associating and translating 
between materials becomes even more complex in class, where individual pupils link up in different ways at 
different paces and with different materials in the chain - and where more artefacts are linked to the system. 
In class, I for instance observe how some pupils prefer to work with the iPad, and others are continually 
occupied with building in Lego. In class, I also observe how one boy, who is very competent at building in 
Minecraft, links his iPad to the class projector which displays his process of building Brandenburger Tor to 
the class. This performance of ‘building Brandenburger Tor’ is an activity, supported by the teacher, that 
takes place simultaneously with other building processes and trajectories through the chain. The generic work 
with building can in this way be constructed individually or specifically, as ‘chains within the chain’ of 
learning through sociomaterial enactment and bricolage. 


7. CONCLUSIONS 

In this paper I have identified ways in which teachers activate iPads and other kinds of resources in learning 
by linking them to systems of their own devising in which artefacts and their modalities constitute significant 
nodes in the web of learning. As a resource in the classroom the iPads participate actively in and in different 
ways contribute to both maintaining established ways of learning and innovating learning through for 
instance the processes of capturing, bricolaging and building. In the examples given, the iPad never acts as an 
isolated actor in learning but is continuously enrolled and made significant in pupils learning as an aspect of 
teachers’ construction of sociomaterial systems of learning. These systems of material learning engage both 
basic tools and technologies such as the iPad, and enable pupils to link onto different parts of the chain. 
Building in this way becomes both a metaphor for learning process itself, and for the ways in which the 
pupils can construct their individual learning trajectories and incorporate or give priority to specific learning 
materials. 

As a flexible, mobile device the iPad may be specifically suited to engaging in fluid and emergent 
teaching and learning practices and to combine with heterogeneous content, modalities and learning rhythms. 
Bricolaging and capturing are examples of such processes where the iPad adds to and innovates the visual 
culture of Middletown school and where modalities are combined in new associations of literacy practices 
(ie. capturing). However, as the examples illustrate, teachers are nonetheless central in initiating, negotiating 
and managing these chains of resources and in understanding how the links between learning activities and 
materials support the learning needs of different learners. The paper therefore suggests that teachers’ digital 
literacy must include not only the ability to use and manage specific educational technologies in the 
classroom but to understand the significance of the ecology of learning, i.e. of the web of relationships 
between resources (including ‘technologies’) and how these resources can be linked to make sense in pupils’ 
learning. Such an approach to teachers’ digital literacy will make room for processes of translation and 
association between the “heterogeneous set of bits and pieces” that make up learning. 
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ABSTRACT 

The purpose of this research study was to investigate if and how a blended approach to Canadian First Nations education 
could be used to foster student engagement and success. The study examined the SCcyber E-Learning Community 
program (2012) through the lens of the Seven Principles of Effective Teaching (Chickering & Gamson, 1999). Data was 
collected via an online survey, interviews, and site visits. The study participants indicated that the deliberate and 
intentional integration of local learning centers and mentors with online teachers, who provide synchronous tutorials 
through the use of a web-based learning management system and conferencing tool, was the key to academic success. 
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1. INTRODUCTION 

The Sunchild First Nation Reserve (40) is located in the western central part of Alberta, Canada. The reserve 
has an area of 52.18 square km. As of 2008, the First Nation has a registered population of 1209 people, of 
whom 732 live on their reserve (Government of Canada, 2008). 

In 1999, members of the Sunchild First Nation considered the lack of education in their community and 
decided alternative methods were needed to reach First Nations students. They discovered that: 

• First Nations students faced unique challenges including family and legal situations, time away from 
class and relocating to new homes. 

• Many First Nations students were adults. These students wanted to upgrade and build a better future 
while meeting their current schedules and responsibilities (SCcyber E-Learning Community, 2012). 

In order to address these challenges the SCcyber E-Learning Community Program was established. This 
program adopted a blended learning approach for high school courses by combining the use of learning 
centers and local mentors with online teachers who provide synchronous tutorials through the use of a web- 
based learning management system and conferencing tool. 


2. BODY OF PAPER 
2.1 Blended Learning 

The idea of blending different learning experiences has been in existence ever since humans started thinking 
about teaching (Williams, 2003). What has recently brought this term into the limelight is the infusion of 
web-based technologies into the learning and teaching process (Clark, 2003). These technologies have 
created new opportunities for students to interact with their peers, teachers, and content. 

Blended learning is often defined as the combination of face-to-face and online learning (Williams, 
2002). Ron Bleed, the former Vice Chancellor of Information Technologies at Maricopa College, argues that 
this is not a sufficient definition for blended learning as it simply implies “bolting” technology onto a 
traditional course, using technology as an add-on to teach a difficult concept or adding supplemental 
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information. He suggests that instead, blended learning should be viewed as an opportunity to redesign the 
way that courses are developed, scheduled and delivered through a combination of physical and virtual 
instruction, “bricks and clicks” (Bleed, 2001). The goal of this redesigned approach to education should be to 
join the best features of in-class teaching with the best features of online learning to promote active, self- 
directed learning opportunities for students with added flexibility (Garnham & Kaleta, 2002). This sentiment 
is echoed by Garrison and Vaughan (2008) who state that “blended learning is the organic integration of 
thoughtfully selected and complementary face-to-face and online approaches and technologies” (p. 148). A 
survey of e-learning activity by Arabasz, Boggs & Baker (2003) found that 80 percent of all higher education 
institutions and 93 percent of doctoral institutions offer hybrid or blended learning courses (Figure 1). 



Figure 1 . Campus-based blended learning approach 

Campus-based environments have their roots in educational systems where classes have been delivered 
by teachers in synchronous class lecture settings. Initially, blended learning has been used to complement 
these synchronous lectures through the use of asynchronous discussion forums and learning management 
systems such as Blackboard and Moodle. With the advent of synchronous tools, such as Blackboard 
Collaborate and Adobe Connect , opportunities have been created to provide students with both synchronous 
and asynchronous communication possibilities. 

ON-CAMPUS ONLINE 


A 
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Power (2008) has coined the term Online Blended Learning to describe the simultaneous and 
complimentary integration and implementation of an asynchronous-mode learning environment (i.e. a course 
management system, or CMS) and a synchronous desktop conferencing environment (i.e. virtual classroom). 
The SCcyber E-Learning Community framework has further expanded this conception of blended learning 
by fully integrating face-to-face and online synchronous and asynchronous learning opportunities for their 
students through the use of local mentors at physical learning sites and highly qualified online teachers 
(Figure 2). 

2.2 Methods of Investigation 

An action research (Stringer, 1999) and case-based method (Creswell, 1997) were utilized for this study. This 
approach consisted of a mixture of quantitative and qualitative data collection methods. All students enrolled 
in the SCcyber E-Learning Community program were invited to complete an online survey in the fall 2011 
semester and then follow-up online interviews were conducted in December 2012 with four of the students 
who completed this survey. In the winter 2012 semester, these online interviews were expanded to include 
seven mentors, two online teachers, and the principal of the program. Two site visits were also conducted 
(Chinki Adult Education Center and the Calgary Aboriginal Futures Center). 

The educational research literature strongly suggests that student engagement is the key to academic 
success and retention (Astin, 1999; Kuh, 2008; Pace, 1980; Pascarella & Terenzini, 2005). The National 
Survey of Student Engagement (NSSE) defines student engagement as the amount of time and effort that 
students put into their academic studies that lead to experiences and outcomes that constitute student success, 
and the ways that programs allocate resources and organize learning opportunities and services to induce 
students to participate in and benefit from such activities (NSSE, 2011). The NSSE is constructed on the 
Seven Principles of Effective Teaching (Chickering & Gamson, 1999): 

1 . encourages contact between students and teachers, 

2. develops reciprocity and cooperation among students, 

3. encourages active learning, 

4. gives prompt feedback, 

5. emphasizes time on task, 

6. communicates high expectations, and 

7. respects diverse talents and ways of learning 

These seven principles are based on over fifty years of educational research (Graham et ah, 2001) and 
they were used to guide the data collection and analysis for this study. 

2.3 Findings and Recommendations 

This section begins with a demographic profile of the student participants followed by a summary of the 
results for each of the three research questions based on the seven principles of effective teaching framework: 

1. What are the advantages of a blended approach to Canadian First Nations education? 

2. What are the challenges? 

3. Recommendations for improving this approach to Canadian First Nations education? 

2.3.1 Demographic and Technology Profile of Student Participants 

There were approximately three hundred students enrolled in the SCcyber E-Learning Community in the fall 
2011 semester. In order to establish a context for the study findings, the initial online survey asked a series of 
demographic questions (n=24, 8% response rate). Table 1 compares the demographics of the SCcyber 
E-Leaming Community students to students at a university in Calgary, Alberta who had recently completed a 
similar survey (Vaughan et ah, 2011). 
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Table 1 . Student Comparison of SCcyber E-Learning Community and Mount Royal University Students 


Student Item 

SCcyber E-Learning Community 

Mount Royal 
University 

Gender 

Female 

68% 

55% 

Male 

32% 

45% 

Age 

Under age of 24 

54% 

89% 

Over age of 24 

46% 

11% 

Place of Residence 

Living with parents 

46% 

62% 

Living with own family with children 

42% 

0% 

Living alone 

0% 

10% 

Living with roommates or partner with no 
children 

13% 

23% 

University residence 

0% 

5% 

Employment Status 

Currently not working 

83% 

23% 

Currently working part-time 

16% 

65% 

Currently working full-time 

0% 

23% 


Two-thirds of SCcyber E-Learning Community students were female and one -third were male. The 
students surveyed ranged in age from 18 to over 41. There appeared to be a bi-modal age distribution with 
two-thirds of the students between the ages of 15 to 27 and one-third between the ages of 31 to 41 plus. In 
terms of place of residence, 46% of the students lived with their parents, 42% lived with their own family 
with children, and 12% lived with roommates or partner with no children. With regards to employment 
status, 83% of the students were not currently working compared to only 23% of the Mount Royal students 
( the remainder have either a part or full-time job). 

A similar comparison was made between SCcyber E-Learning Community and Mount Royal University 
students with regards to access to technology and self-reported skills (Table 2). 


Table 2. Comparison of Technology Access and Skills between SCcyber E-Learning Community and Mount Royal 

University Students 


Technology Item 

SCcyber E-Learning 
Community 

Mount Royal 
University 

Home access to the Internet 

37% 

100% 

Access to high-speed home Internet connection 

33% 

98% 

Have your own a mobile communication device 
(e.g., cell phone) 

62% 

90% 

Have your own laptop computer 

38% 

89% 

Have your own a mobile communication device with Internet 
access (e.g.. Smart Phone) 

29% 

82% 

Personal Rating of Computer Skills 

Novice (not really comfortable using computers) 

0% 

5% 

Intermediate (comfortable using computers) 

70% 

59% 

Advanced (have developed some expertise and enjoy 
using a computer) 

30% 

36% 
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In terms of technology access, only 37% of the SCcyber E-Learning Community students surveyed had 
home internet access. Two-thirds of the students had a mobile communication device (e.g., cell phone. 
Blackberry, iPhone), 38% had their own laptop, and 33% had access to a desktop or laptop computer at home 
that they share with others. Despite this lack of home and personal access to computer technologies, through 
participation in the SCcyber program, 70% percent of the students rated themselves as intermediate with 
regards to their computer skills while 30% rated themselves as experts. 

2.4 Seven Principles of Effective Teaching Practice Framework 

Below are the seven principles analyzed qualitatively in regards to the successful framework in which 
Sccyber E-learning model operates and highlights how this has resulted in successful course completion 
rates. 

2.4.1 Principle 1: Good Practice Encourages Student-Teacher Interaction 

Synchronous and asynchronous communication technologies are being used by students in the SCcyber 
E-Leaming Community to increase access to their online teachers and mentors, help them share useful 
resources, and provide for joint problem solving and shared learning that is being combined with face-to-face 
mentoring at the learning centers. These communication technologies are strengthening online teacher 
interactions with all students, but especially with shy students who are reluctant to ask questions or challenge 
the teacher directly. These students find that it is often easier to discuss values and personal concerns in 
writing rather than orally, since inadvertent or ambiguous nonverbal signals are not so dominant. 

The roles and responsibilities of the online teacher in this program can become overwhelming and a 
recommendation has been made to have each of the online teachers log their daily activities for a one week 
period. Then, at one of the monthly team meetings the results can be shared and strategies developed for 
managing the workload of an online teacher in the SCcyber E-Learning Community. 

2.4.2 Principle 2: Good Practice Develops Reciprocity and Cooperation among Students 

The SCcyber E-Learning Community strategically works at creating a cooperative learning environment 
amongst the students, parents, mentors, and online teachers. The focus of the program is on self-paced 
learning but the study participants suggested that communication and information technologies could be used 
to support additional opportunities for study groups, collaborative learning, group problem solving, and 
discussion of assignments. 

In addition, many of the students and mentors emphasized how important it is to create a sense of 
community at the learning centers (e.g., displaying student work on the walls, creating a student council, 
creating a lunch and leisure space). A recommendation has been made to have senior mentors travel to new 
sites to help the new mentors establish their learning centers. 

2.4.3 Principle 3: Good Practice Uses Active Learning Techniques 

The range of technologies that the SCcyber E-Learning Community uses to encourage active learning is 
extensive. In the past, apprentice-like learning has been supported by many traditional technologies: libraries, 
laboratories, art and architectural studios, athletic fields. Newer digital technologies can now enrich and 
expand these opportunities - especially for those students located in rural and remote parts of Alberta and the 
Northwest Territories. For example: 

• Supporting apprentice -like activities in fields that themselves require the use of technology as a tool, 
such as statistical research and computer-based music, or use of the Internet to gather information 
not available in the local library. 

• Simulating scientific techniques such as helping chemistry students develop and practice research 
skills in “dry” simulated laboratories. 

• Helping students develop insight. For example, students can be asked to design a radio antenna. 
Simulation software displays not only their design but the ordinarily invisible electromagnetic 
waves the antenna would emit. Students change their designs and instantly see resulting changes in 
the waves. The aim of this exercise is not to design antennae but to build deeper understanding of 
electromagnetism. 
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Many of the students enrolled in this program also have their own mobile devices and a recommendation 
has been made to have them use these devices to document and record their learning in their local 
communities. For example, they could use their phones to take pictures and record videos that could then be 
used in the creation of digital stories for course assignments ( Center for Digital Storytelling - 
http://www.storycenter.org/). 

2.4.4 Principle 4: Good Practice Gives Prompt Feedback 

The combination of a learning center mentor and online teacher for each course ensures that all students 
enrolled in the SCcyber E-Learning Community receive timely and regular feedback about their academic 
studies. Students receive a report card every Monday morning and they can make adjustments to their work 
level for course completion on a weekly basis. Computers also have a growing role in recording and 
analyzing personal and professional performances. Teachers can use technology to provide critical 
observations for an apprentice; for example, video to help a novice teacher, actor, or athlete critique his or 
her own performance. Teachers (or other students) can react to a writer’s draft using the “hidden text” option 
available in word processors: Turned on, the “hidden” comments spring up; turned off, the comments recede 
and the writer’s prized work is again free of “red ink.” 

In addition, as Alberta Education moves toward portfolio assessment strategies, computers can provide 
rich storage and easy access to student products and performances. Computers can keep track of early efforts, 
so teachers and students can see the extent to which later efforts demonstrate gains in knowledge, 
competence, or other valued outcomes. Performances that are time-consuming and expensive to record and 
evaluate — such as leadership skills, group process management, or multicultural interactions — can be 
elicited and stored, not only for ongoing critique but also as a record of growing capacity. 

2.4.5 Principle 5: Good Practice Emphasizes Time on Task 

The SCcyber E-Learning Community program allows students to work at their own pace in a safe 
environment with constant monitoring of their progress. The mentors and online teachers interviewed 
indicate that some students have problems completing their assignments in a timely fashion and thus, have to 
hastily complete a large portion of them at the very end of the semester. Strategies have been put in place to 
enforce regularly-distributed deadlines that encourage students to spend time on tasks and help them avoid 
procrastination. These deadlines also provide a context for regular weekly contact with the mentors and 
online teachers. 

2.4.6 Principle 6: Good Practice Communicates High Expectations 

This program does an excellent job of communicating high expectations and publicly praising students 
through the Wall of Success (student course completion certificates) at each learning center. Communicating 
high expectations for student performance is essential. An additional way for teachers to do this is to give 
challenging assignments. For example, assigning tasks that require students to apply theories to real-world 
situations rather than remember facts or concepts. This case-based approach involves real-world problems 
with authentic data gathered from real-world situations. 

Another way to communicate high expectations is to provide examples or models for students to follow, 
along with comments explaining why the examples are good. Teachers can provide examples of student work 
from a previous semester as models for current students and include comments to illustrate how the examples 
met the required expectations. In addition, the online teacher can provide examples of the types of 
interactions she or he expects in the discussion forum. One example would be to provide an exemplary 
posting while also providing an example of what not to do, highlighting trends from the past that she or he 
would like students to avoid. 

2.4.7 Principle 7: Good Practice Respects Diverse Talents and Ways of Learning 

Finally, the SCcyber E-Learning Community clearly demonstrates how communication and information 
technologies can be used to support different methods of learning through powerful visuals and 
well-organized text; through direct, vicarious, and virtual experiences; and through tasks requiring analysis, 
synthesis, and evaluation, with applications to real-life situations. These digital tools are also being used to 
encourage self-reflection and self-assessment. In addition, technologies are being used in this program to 
help students learn in ways they find most effective and broaden their repertoires for learning. The 
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technologies, with the mentor and online teacher’s support, are supplying the structure for students who need 
it while leaving assignments more open-ended for students who don’t. Fast, bright students can move quickly 
through materials they master easily and go on to more difficult tasks; slower students can take more time 
and get more feedback and direct help from the online teachers and mentors. 


3. CONCLUSION 

If students learn to make education a priority they are going to succeed in life. 

( SCcyber Mentor Inten’iew) 

The study participants indicated that the blended approach of the SCcyber E-Learning Community program 
through the deliberate and intentional integration of local learning centers and mentors with online teachers, 
who provide synchronous tutorials through the use of a web -based learning management system and 
conferencing tool, was the key to academic success. They also emphasized how this blended approach 
helped First Nations students overcome major learning challenges such as remote locations, lack of access to 
digital technologies, high speed internet access, and quality teachers. 

Finally, every SCcyber E-Learning Community student who participated in this study commented on the 
“passion and commitment’’ that the mentors, online teachers, and administrators involved in this program had 
for student success. They all emphasized that the SCcyber E-Learning Community was “making a difference 
for their lives”. This enthusiasm for learning is definitely infectious and it is strongly recommended that more 
government departments, educational institutions, and corporations partner with this program in order to 
expand the positive impact on the lives of First Nations students in Canada. 
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ABSTRACT 

The purpose of this paper is to describe a learning model based on Storytelling and its application in the context of legal 
education helping build challenging training resources that explain, to common citizens with little or no background 
about legal topics, concepts related to Legal Mediation in general and in specific areas like e-commerce and civil liability. 
The defined model has been contextualized with respect to relevant literature and implemented through the development 
of two software components that have been integrated in an existing e-learning environment. Such an e-learning 
environment is itself a module of a greater experimental system for on-Line Legal Mediation named eJRM. 
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1. INTRODUCTION 

In March 2011 the Italian Government introduced mandatory pre-trial mediation of civil and commercial 
cases. After a ten-month stop due to a provision by the Italian Constitutional Court, the mandatory mediation 
came in force again in September 2013 with a new regulation. The Italian mediation model is capable of 
sensibly speed-up the settlement of disputes but, on the other end, it needs efficient and effective tools to 
support the explosion of mediations. Citizens also need to be sensitized to the benefits of mediation and must 
be trained on how mediation works and how to access it. 

The project eJRM “ electronic Justice Relationship Management ”, supported by the Italian Ministry of 
University and Research, brings together researchers and practitioners in the fields of law and computer 
science with the aim of defining, implementing and experimenting innovative methods for managing on-line 
mediations. The project foresees the development of a complete environment for on-line mediation with 
innovative features like the possibility, for a citizen, to formalize a case in natural language and let the system 
provide relevant information for autonomous case resolution such as connected laws, relevant jurisprudence, 
training modules, links to lawyers with useful competencies, etc. (Arosio et al., 2013). 

eJRM also covers the definition and development of engaging training modules, targeted to citizen with 
little or no background on legal topics, aimed at explaining concepts related to legal mediation in general and 
in specific areas like e-commerce and civil liability. In line with a tradition of Narrative Pedagogy applied to 
legal education, we decided to adopt, in this case, a Storytelling paradigm. The purpose of this paper is to 
describe the developed model and the first results obtained in its implementation. 

Starting from the well-known Visual Story Portrait (Stanley and Dillingham, 2009), we defined a story 
model composed of several situations where each one is made of subsequent events aiming respectively at 
interesting the learner, providing the required information, supporting reflection and assessing the acquired 
knowledge. We also integrated a dynamic branching mechanism that allows the re-articulation of the story on 
the basis of assessed training results to support the recovery of found knowledge gaps. The model has been 
implemented and integrated within an existing training environment. A sample story about mediation in the 
e-commerce field has been also defined for a first validation of the system and of the underlying model. 
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The paper is organized as follows: section 2 contextualizes the research described in this paper in the 
framework of research on narrative pedagogy and storytelling applied to legal matter; section 3 describes the 
defined model; section 4 presents the developed prototype; section 5 describes the experimental story defined 
for system validation. 


2. RELATED WORK 

The goal of legal education is the teaching and learning of legal and case law doctrine. In teaching practices 
documented in literature a clear dichotomy emerges between a more theoretical approach to knowledge 
transfer and a more practical approach aimed at developing skills and knowledge in action. The first, more 
common in Europe ( where law schools are based on traditional methods which make use of resources such as 
lectures, essays and manuals), focuses on the discussion of abstract concepts and on the explanation of rules 
often far from concrete experience. 

Quite different instead is the approach adopted by Anglo-American countries, where the teaching 
approach applied is based on practice, and on assignment of specific tasks to groups of students, pointing out 
resources analysis, and resolution of cases. Most of such schools resort to specific teaching methods like 
Casebooks based on true or fictional stories; Legal Clinics i.e. real hands-on legal experiences where 
students are called to offer pro bono legal services to real clients under the guidance of experienced teachers; 
Moot Courts i.e. simulation of a real appeal and competition with others, whose value is amplified by the use 
of information and communication technologies (Lettieri et al., 201 1). 

Narrative Pedagogy has an increasing importance in this context. Effectively utilized in several 
disciplinary contexts and domains, it guarantees, even within the legal framework, a high degree of learner’s 
involvement and of skills development (Blissenden, 2007). Jurists of Anglo-American countries were the 
first to emphasize the importance of narrative pedagogy and the interrelationship between law and literature. 
Cardozo in deed published the essay “Law and Literature ” already in 1925, introducing the possibility that 
narrative offers to accept, through the literary representation, the context within which legal experience takes 
place; law in action as opposed to law in books. 

However, it is from the 1970s that the researchers began to focus not more on literary texts analysis, but 
on the training and education potential that the use of narration can have in the legal field. Between the 1980s 
and 1990s, several professors from important law schools and also legal scholars, tried to emphasize and 
promote the introduction of Storytelling as an alternative teaching method or as an addition to traditional 
techniques used to teach legal topics. This second phase was launched by studies like “77ze Legal 
Imagination: Studies in the nature of the Legal Thought and Expression'” (White, 1985). 

It is precisely in the 1980s and 1990s that the academic world reassessed the use of narrative in teaching 
(Bruner, 1984) and law schools introduced storytelling in their classrooms. In this way, the understanding of 
concepts that are often too abstract was facilitated through realistic o real stories. Furthermore, students were 
involved in role-playing games that, through different views, allowed them to empathize with the characters 
in the story, creating emotional and empathetic immersion. 

In the period 2007-2009, the literature presented a view of the applied strategy of storytelling in the legal 
environment and “fo weave the law in the stories ” seemed to become the best way to preserve concepts and 
think about possible applications (Steslow and Gardner, 2011). Five techniques to integrate storytelling with 
traditional legal education were defined: Metaphorical Stories (i.e. stories told to explain complex concepts). 
War Stories (i.e. stories told to explain a legal rule or doctrine after the students labour through analysis of 
rule and its application). Case Stories (i.e. stories describing legal cases). Storytelling through literature (i.e. 
lifelike characters and stories from literature taken as a model). Sharing stories (i.e. individuals that integrate 
and build together their own stories expanding them routes and viewpoint). 

In recent years, thanks to digital technologies, the Digital Storytelling captured the attention of experts 
and researchers and leaded to evaluating the potentiality related to its adoption. For example. Civil Obiezion! 
(Steslow and Gardner, 2011) is a series of computer gaming trial simulations in which students play the role 
of lawyers. The players are invited to evaluate questions posed to a witness in the course of a hearing and to 
decide whether it is possible to oppose according to the Federal Rules of Evidence. The student must choose 
when to oppose and select an explanation for the objection made. Acceptance or rejection of the objection are 
decided by computer that plays the role of judge. 
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TLE (Transactional Learning Environment), developed in 2000 by Glasgow Graduate School of Law 
(GGSL) and UK Centre for Legal Education (Maharg, 2007), is a virtual simulation environment used in a 
professional legal practice training course. TLE is built around an on-line virtual town containing utilities, 
businesses, agencies, government organizations etc. which provide the backdrop for students to progress 
simulated legal transactions. 

Fishbowl Online Role-Play (Douglas and Johnson, 2010) is an educational game based on problem 
solving for the development of legal skills. The game provides students with opportunities to practice legal 
skills by taking different roles that allow them to practice interviews to witnesses, legal advice and moments 
of judicial negotiation and enjoy instructional scaffolding moments that promote the construction of 
knowledge in action. The narrative game allows the development of specific skills in the negotiation and 
mediation field as well as those required to conduct an interrogation. 


3. THE LEARNING MODEL 

This section describes the Storytelling model we have defined in the context of the eJRM project. The model 
was purposed to build challenging training resources to explain, to common citizens with little or no 
background about legal matter, concepts related to Legal Mediation in general and in specific areas like e- 
commerce and civil liability. The training resource had to transfer notions about procedures to follow, actors 
involved, normative references about mediation and advantages of such a method of disputes resolution with 
respect to a legal trial. 

To develop our model, the Visual Story Portrait (VSP) was investigated. It is a very much in use story 
map characterized by some essential elements as shown in figure 1: Problem , transformations of main 
character (whom the student is normally identified with) and story closure. To assess the power (the 
storyability) of situations, the story is conceptualized in terms of transformations formations , transformation 
layers of several types: physical/kinesthetic, inner strength, emotional, moral, intellectual, psychological, 
social and spiritual (Ohler, 2006). 

The proposed model considers the intellectual transformation of VSP as changes in terms of learning 
objectives extracted from Bloom's taxonomy (Bloom et ah, 1956). The learners are encouraged to use the 
knowledge and cognitive skills to go through different teaching situations associated with various phases of a 
classical VSP (Mangione et al., 2013). The correspondence between situations of VSP and learning 
objectives of Bloom (Rrathwohl, 2002) guides the transformation of the character and the structure of 
situations. 
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Figure 1. Mapping between VSP and learning objectives. 
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To ensure the achievement of assigned learning objectives, each situation presents itself as the 
composition of educational events whose structure facilitates organization, selection and integration of 
information (Gaeta et al., 2014): the Advancer Event designed to activate student’s prior knowledge and to 
ensure his involvement in the initial situation; the Learning Event that supports the objective to maximize 
student’s topic understanding and is based on a heavily driven approach; the Reflective Event designed to 
support the learner in the reflection process on concepts learned and helping he consolidate the knowledge 
acquired; the Assessment Event able to assess whether the type of cognitive transformation hoped for him has 
occurred. 

By recognizing the knowledge gained by the user in different teaching situations, it is possible to define a 
series of treatments able to support the student in overcoming the presented shortcomings. Assessment Events 
are in fact the driver of a dynamic branching mechanism that allows a re-articulation of the story to recovery 
found knowledge gaps. 

Actually, the defined model supports different "corrective” paths that meet specific educational 
approaches or principles. The alternative routes call the student to take up a new perspective in the story. 
Coherently with an interactionist perspective one accepts the idea of how individuals are able to take certain 
“views” (e.g.. Helper, Hero, Victim, Antagonist, Essay, etc.) during the interaction with the learning resource 
which, selected randomly, allows learners to experience situations and events with greater responsibility and 
empathy (Porteous et al., 2010). 

Figure 2 shows the flow of events within a modelled situation, based on two teaching iterations. The 
assessment event that occurs at the end of a situation, allows to obtain a measure of understanding of key 
concepts through a formative evaluation able to represent a specific goal of knowledge (first iteration). Once 
you have a score (second iteration) the narrative branches out and suggests to the student different alignment 
and recovery paths, basing on specific rules defined by the teacher and related to knowledge levels detected 
by formative tests (Mangione et al., 2011). 



Figure 2. The defined Storytelling teaching flow. 


4. THE DEVELOPED PROTOTYPE 

In order to build and experiment a Storytelling Learning Resource based on the defined model, two software 
components have been defined and integrated within an existing e-learning environment. Such a learning 
environment, named IWT ( Intelligent Web Teacher ), allows the definition and execution of personalized e- 
learning experiences tailored to learners’ cognitive state and learning preferences (Capuano et al., 2008; 
Capuano et al., 2011). In particular, defined software components have been obtained as a customization, in 
the legal domain, of results coming from a previous research initiative oriented to risk management (Capuano 
et al., 2013). 

The first component developed is the Storytelling Editor. It allows the creation of multimedia resources, 
the design of story elements, the creation of testing activities and the management of different flows inside the 
story. It is conceived as a desktop application allowing to select several editing layouts and to support content 
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editing functions. It enables the authors to make creative contents by integrating multimedia objects such as 
textbox, images, video and audio as well as to associate interactive behaviours to each object. A table of rules 
allows to check scores obtained by the learner within each Assessment Event and, if the score reach a given 
threshold, the story flow proceeds to the next situation; otherwise the situation is restarted with different 
parameters (change of media, change of scenario, change of role) according to the process shown in figure 2. 

Figure 3 shows the main interface of the editor. The left pane lists the sequence of situation events while 
the selected event is displayed in the content pane on the right for editing. The toolbar on top of the window 
includes editing functions and allows to insert, in the current event, several kinds of multimedia objects. A 
recording toolbar is also provided on the right to capture live audio and video. 



Figure 3. Screenshot of the Storytelling Editor. 

The second component is the Storytelling Player, a Web application based on Microsoft Silverlight that can 
execute the story according to a designed flow. By interacting with scenes during the story, the learner returns 
a feedback to the player, while the underlying story engine, basing on them, can reason on the right 
continuation of the story. Figure 4 shows a screenshot of the storytelling player integrated within the learning 
environment. On the left pane there is the sequence of situation events while on the right pane the current 
event is displayed. The learner interacts within the main pane and the flow of events changes accordingly with 
respect to defined rules. 



Figure 4. Screenshot of the Storytelling Player. 
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5. AN EXPERIMENTAL STORY 

In the context of eJRM, several stories based on the model described in section 3 are currently under 
development. We present here a first experimental story, covering the topic of mediation in the e-commerce 
field that we have defined to validate the prototype as well as the underlying model. The didactic objective is 
articulated into three sub -Objectives: understanding the concept of mediation and situations; understanding 
the life cycle of mediation and key roles for its correct management; opportunities and possible consequences 
of mediation. The story flow is graphically depicted in figure 5 while a screenshot of the story within the 
Storytelling Player is shown in figure 4. 

Every situation of the learning resource answers a specific didactic transformation path based on Bloom’s 
taxonomy. The first situation ( Introduction ), presents an Advancer Event, where the protagonist, Paola, 
following numerous commercials of the e-commerce site Orelejo.it broadcast on TV, decided to make her 
first online purchase, buying a watch with an automatic mechanism. In the Learning Event, it is possible to 
learn key concepts such as methods of concluding contracts online and applicable law discipline, e -payment 
systems, and personal data protection. The Reflection Event clarifies the conclusion of the online contract and 
finally an Assessment Event proposes questions and multiple choice test to understand risks and advantages 
of online shopping. 

The second situation ( Call of Adventure), in the Advancer Event, proposes a scene where Paola receives 
the clock and, after checked for its proper operation, wears it, and immediately starts using it. Something 
goes wrong so, only two months after the purchase, the automatic mechanism is blocked. Paola is 
disappointed and tries to solve the problem by turning the crown for manual charging many times and 
strongly, but the watch does not work anymore. Paola, not knowing what to do, returns to the website where 
the purchase was made and reads about the possibility to withdraw from the contract as well as information 
on product’s warranty. The Learning Event focuses on concepts of consumer protection, conditions, timing 
and withdrawal rule exercising procedures, and sale’s guarantees. A Reflection Event on the importance of 
consumer protection follows as well as an Assessment Event on how to fill the request for product 
replacement and cases in which you can take advantage of the replacement procedure. 

In the third situation (Problem), the Advancer Event, Paola learns all the needed information that will help 
her understand that she can make use of a warranty covering her watch purchase. Paola, therefore, 
communicates the problem to the vendor and requests to have it replaced. After a considerable period of 
time, the vendor replies communicating to Paola that the company does not intend to replace the watch 
because, after having analysed the problem, they found the failure to be caused by an incorrect operation 
made by Paola. She, in fact, in an attempt to unlock the watch, would have forced the watch crown, causing a 
mechanism fault. 

In the seller’s opinion, Paola should have immediately asked for a watch replacement, as soon as the 
problem had occurred, without making any arbitrary attempts. The seller, however, is willing to repair the 
clock at Paola’ s expense. Paola does not agree with this solution and insists on her right to replacement. In 
the Learning Event, times and conditions of guarantee are explained as well as the consumer options, 
including replacement and repair of goods; a price reduction or termination of contract. After a Reflection 
Event on the discipline of guarantee, understanding of the explained concepts is assessed through an 
Assessment Event. 

The fourth situation (Middle), in the Advancer Event, sees Paola contacting a lawyer, who clarifies her 
that, according to the Legislative Decree no. 28/2010, as recently amended, even if e-commerce is an area not 
covered by mandatory mediation, there is the option to do so anyhow, trying to avoid arbitration, as well as 
issues related to it. Paola searches for additional information on the Internet and discovers the existence of 
eJRM, a system that guides citizens in legal mediation. By this website, Paola does a self-assessment, she 
shortly describes her case, the system then classifies the case and suggests her to undertake the process of 
mediation. Through the site, Paola also understands how the mediation is performed and finds similar cases 
resolved through mediation. The Learning Event presents resources that help learn more about the current 
discipline regulating mediation for civil and commercial disputes, together with the concept of voluntary and 
mandatory mediation, and key figures in the process of mediation. After a Reflection Event on key concepts, 
the Assessment Event requires the student to identify and sort pros and cons of mediation compared to the 
legal action. 
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The fifth situation (Solution), in the Advance Event , introduces the online mediation process and identifies 
the expert profile for similar cases and the preparation of the first meeting. Methods of preparation and 
presentation of on-line mediation request are detailed during the learning event, and after a Reflection Event 
about managing mediation meetings, a simulation-based Assessment Event requires the user to identify the 
best way to manage a mediation meeting. 

In the sixth situation, (Closing), the Advancer Event shows a virtual room meeting, where the mediator is 
unable to find a solution between the parties who in the end will go to a trial. The Learning Event emphasizes 
the importance of the mediation process, and after a Reflection Event , the user is asked to go through an 
Assessment Event to understand and identify ways of managing an event in order to reach an agreement. In 
this latter event, in case the learner does not reach a satisfactory score, the system suggests a micro adaptive 
recovery path where the user is given an alternative scene where through various steps he can achieve a 
successful mediation. A new facilitated Assessment Event, will then test the successful knowledge 
acquisition. 


Situation 1 : Introduction 

Paola buys a watch on a e-commerce site 

Learning goals: online contracts, e-payment 


Situation 2: Call ot Adventure 

After two months the watch stops working 

Learning goals: consumer protection, guarantee 


Situation 3: Problem 

The seller refuses to replace the watch 

Learning goals: times and conditions of the guarantee 


Situation 4: Middle 
Paola discovers the possibility to mediate 
Learning goals: mediation discipline, on-line mediation 


Situation 5: Solution 

Paola asks the seller to mediate 

Learning goals: how to present an on-line mediation request 


Situation 6: Closure 

A mediation meeting is organized without finding an agreement 
Learning goals: what happen in a mediation meeting 


Situation 6: Closure ( re-iteration l 

A mediation meeting is organized and an agreement is found 
Learning goals: what happen in a mediation meeting 


Figure 5. The teaching flow of the experimental story. 


6. CONCLUSION 

We described in this paper a new learning model, based on narrative pedagogy and storytelling, that 
integrates dynamic branching facilities allowing re-articulation of the story according to assessed training 
results. The model has been instantiated in the legal domain to build engaging learning resources about 
online mediation, targeted to users with limited background on legal topics. A prototype and a sample story 
have also been defined and developed to provide a first empirical validation of the model. The prototype was 
shown to a group of teachers expert of legal education and some trial scenarios have been simulated. 

The results obtained up to now are encouraging and confirm that the storytelling model is capable of 
building valuable learning resources in the field of legal education. The defined model is able to represent a 
complex storyboard covering the topic of mediation in e-commerce while the developed prototype is capable 
of editing and playing multimedia components connected to each story event and situation as well as to 
model the underlying teaching flow. 

The obtained learning resource is a fully interactive didactic element oriented to a real learner centred 
educational approach and able to provide guidance and to make the reflection easier. The transmedia nature 
of the obtained learning resources allows the creation of an augmented narrative that is capable of supporting 
learners participation and enhancement of concepts and skills. 

The work described in this paper is still on-going in the framework of the eJRM project. Additional 
stories are currently in course of definition together with a complete experimentation and validation plan with 
real users, involving an Italian mediation entity and a selection of real customers. The validation will follow 
the expert-based approach combined with an empirical one. In either cases we will apply an analysis 
purposed to the verification of both learning results as well as the learner’s involvement level. 
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ABSTRACT 

The effective training of users is a key factor of the success of Enterprise Resource Planning (ERP) system projects. This 
need for ERP system training is exacerbated by a demand for quality ERP consultants which is evident in Europe and in 
African countries, particularly in South Africa where science and technology education has been identified as a national 
priority. However, the high costs of traveling in developing countries within Africa can make it difficult to offer 
traditional face to face training by vendors. Mobile learning or m-learning has shown a significant rise in use by 
companies and researchers developing learning platforms. M-learning can be used to aid in the training of ERP users who 
are on the go as well as those who do not have direct access to desktop PCs in their work environment. Whilst several 
studies propose the use of m-learning systems, research related to the implementation and evaluations of m-learning 
systems which focus on ERP system education are limited. In this paper success factors for m-learning of ERP systems 
are identified. An understanding of these factors can improve the design of an ERP m-learning system, thereby 
facilitating an improvement in ERP and more broadly speaking, technology education. This paper reports on field studies 
where the Technology Acceptance Model (TAM) was used to evaluate the acceptance, usefulness and perceived ease of 
two systems, the OpenSAP e-learning application and the SAP Learn Now m-learning application. The study found that 
the m- learning system was accepted by the learners and was rated positively for perceived ease of use (PEOU) and 
perceived usefulness (PU). The study results also showed that the TAM model could be successfully used to evaluate e- 
leaming and m-learning systems. 


KEYWORDS 

m-learning; ERP system education; e-learning. 


1. INTRODUCTION 

There has been a global increase in the number of small to medium size organisations making use of 
Enterprise Resource Planning (ERP) systems (Zhang, Gao, & Ge, 2013). ERP systems are a suite of business 
modules or applications that integrate the information and processes of the various business units within an 
organisation, such as finance, marketing and production. The integration of these units streamlines the flow 
of information which can bring benefits such as process improvements and reduction of inventory costs into 
the value chain (Beheshti, 2006). However these benefits cannot be achieved if the implementation of an 
ERP system is not successful. The total cost of ERP implementation failure is high and is rising rapidly 
(Momoh, Roy, & Shehab, 2010). Galy and Sauceda (2014) argue that the training of users of an ERP system 
is the corner-stone of the implementation. Several other studies have also cited ERP training as a key factor 
in ERP system implementation success (Bologa & Lupu, 2014; Dalveren, 2014; Dezdar, 2012). Other factors 
affecting ERP system implementation success are the demand for ERP skills and knowledge and access to 
graduates with industry-relevant ERP education (Bologa & Lupu, 2014; Scholtz, Calitz, & Cilliers, 2011). A 
demand for ERP skills has resulted in the introduction of ERP system courses in higher education degree 
programmes (Dalveren, 2014). It is therefore important to have well-structured education programmes at 
universities as well as industry training programmes for users and other stakeholders in ERP system 
implementations (Ram, Wu, & Tagg, 2014). In this paper we refer to ERP learners as those either in 
education programmes in universities or ERP users in industry training programmes. 


37 



ISBN: 978-989-8704-08-5 © 2014 


It is evident that there is a need to address research regarding the design of ERP training and education 
programmes in order to find the most effective and efficient forms of delivery and reduce the time and cost 
involved whilst still maintaining the required quality. ERP training programmes can be presented through 
different forms varying from the traditional face to face courses to long distance learning platforms such as 
electronic learning (e-learning) and mobile learning (m-learning). Modern organisations have become more 
inclined to e-learning programmes due to the benefits of differing work schedules of employees and 
differences in geographical locations between vendors and the organisations implementing the ERP system 
(Pastor & Casanovas, 2002). E-learning is the use of technology to aid in learning at any place and anytime. 
M-learning, refers to the use of mobile technologies to support learning (Attewell & Savill-smith, 2003). The 
2013 Africa E-Learning Report has identified mobile technologies as an important contributor to learning 
technologies in Africa in the last five years (Isaacs, Hollow, Akoh & Harper-Merrett, 2013). 

The research field of m-learning has attracted interest from researchers as well as from companies 
developing and implementing learning systems (Uzunboylu, Cavus, & Ercag, 2009). While m-learning has 
attracted the interest of many researchers and organisations looking to implement a more flexible long 
distance learning environment, very little research has been done in using m-learning systems within the ERP 
industry. For this reason this paper proposes an m-learning model of success factors for ERP e-learning and 
m-learning education and training programmes. This paper is structured as follows: The related literature in 
the areas of e-learning and m-learning are discussed in Section 2. The research methodology used is 
discussed in Section 3 and the results are analysed and discussed in section 4. Finally the recommendations 
and conclusions are addressed in Section 5. 


2. E-LEARNING AND M-LEARNING SUCCESS FACTORS AND ERP 
EDUCATION 

2.1 E-Learning in the ERP Industry 

Several definitions of e-learning have been proposed. According to Sharma and Kitchens (2005) e-learning 
includes learning with web-based training facilities such as virtual universities and classrooms which allow 
for digital collaboration and technology assisted distance learning. Siqueira, Braz and Melo (2007) define e- 
learning as the use of Internet related technology in education or training. E-learning has an advantage over 
traditional learning in that apart from it having the ability to break geographical barriers it also offers the 
potential of a more flexible, tailor-made learning environment which can adjust according to both to the 
learners’ knowledge and skills and also to their preferred learning style (Sambrook, 2003). 

Although e-learning has gained popularity it is important to be aware of factors affecting the success of e- 
learning projects (Table 1). One of these factors is the quality of content material, which can impact the 
success of an e-learning project (Siqueira et al., 2007). Nedungadi and Raman (2012) further suggest that the 
appropriate management of the content is important to ensure that e-learning is efficient. This management 
involves the development and maintenance (updates) of relevant information and content, which is time 
consuming and expensive. In order to address this issue the learning process must be as quick and as cost 
effective as possible whilst still achieving the learning goals. This is attained through “rapid e-learning” 
which reduces the duration of e-learning projects (Unneberg, 2007). By enabling those producing the e- 
learning content, establishing a well organised assessment and approval process and ensuring the training is 
consistent across the organisation this can be achieved. Hawking and McCarthy (2006) propose an e-learning 
model that integrates synchronous and asynchronous learning approaches in order to assist the delivery of 
offshore ERP education. Asynchronous e-learning does not involve the presence of a teacher; rather learning 
content is located on a remote server which learners can access over the Internet. However, synchronous e- 
learning requires e-learning to be conducted with the learner and the teacher being present at the learning 
event at the same time. This model has been adopted by most ERP vendors in that they have learning 
community sites where various learners can access information as well as interact with instructors or experts 
in particular knowledge areas who provide assistance as well as virtual classes (Hawking & McCarthy, 2006). 
E-learning offerings should include training, delivery of information almost immediately and consultancy 
from experts (McGill, Klobas, & Renzi, 2014) . 
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Table 1. Factors for improving the success of e-learning projects. 


Factor 

System 

Author 

Fast access to course material 

Higher Education 
System 

McGill, Klobas, and Renzi (2014) 

Combination of synchronous and asynchronous 
e-learning applied 

ERP system 

Hawking and McCarthy (2006) 

Quality of content 

Data Warehousing 
System 

Siqueira et al. (2007) 

Content modelling 

Data Warehousing 

Siqueira et al.(2007) 


System 

Nedungadi and Raman (2012) 

Enabling of those producing e-learning content 

Marketing System 

Unneberg (2007) 

Consultancy from experts 

Higher Education 

McGill, Klobasand Renzi (2014) 


System 

Hawking and McCarthy (2006) 

Online certifications 

ERP system 

Dalveren (2014) 

Provision of instructor videos 
Interactive learning assignments 
Assessments (theory and practical) 


Hawking and McCarthy (2006) 


Several ERP vendors such as SAP, SYPRO, SAGE and Microsoft have introduced e-learning platforms 
to meet the modern demands of the customers for an e-learning platform (Dalveren, 2014; Mishra & Mishra, 
2011). These solutions offer learners access to an ERP e-learning platform whereby they have unlimited 
access to online training materials, online certification as well as training and learning plans. SAP’s 
innovative e-learning platform, openSAP, enables learners, SAP professionals, developers, entrepreneurs and 
consultants the opportunity to learn about SAP and its innovations (SAP, 2013). It also gives the learners an 
opportunity to interact with other learners and SAP experts on subject matters. The openSAP e-learning 
system consists of various courses taught on ERP and SAP and consists of a number of units offered at 
different times throughout the year. However e-learning may not be ideal for all situations even though it is 
well implemented. The 2013 e-learning Africa Review reported finances and hardware costs amongst the 
constraints of e-learning. Additional bandwidth and electricity costs of maintaining desktop computers were 
also cited (Isaacs, et al., 2013). 

2.2 M- Learning and ERP Education 

The 2013 e-learning Africa Review identifies mobile technologies as the leading driving technologies in 
education in Africa (eLearning Africa 2013 : In Review , 2013) The report identifies the low cost of mobile 
devices as compared to desktop PCs and bandwidth amongst the reasons for mobile devices being the 
preferred technologies. Connectivity on mobile devices is significantly cheaper than on PCs and this may be 
more appealing to users in developing countries and those who may not have PCs at their homes (Nordin, 
Embi, & Yunus, 2010; Uzunboylu et al., 2009). M-learning has therefore been deemed as the new paradigm 
in education. According to Nedungadi and Raman (2012) the progression of distance learning and training 
can be characterised as the move from distance learning to e-learning to m-learning. M-learning involves the 
use of handheld mobile devices to facilitate and enhance the learning process (Nedungadi & Raman, 2012). 
M-learning can also be defined as the use of wireless phones or personal digital assistants to deliver digitised 
content (Sharma & Kitchens, 2005). Furthermore, m-learning refers to learning conducted in multiple 
locations, across multiple times with content accessible from various devices, for example smartphones and 
tablets (Oberer & Erkollar, 2013). M-learning provides collaborative interaction and learning opportunities 
for persons and groups that are geographically dispersed (Uzunboylu et al., 2009). 

Several factors for m-learning project success have been identified (Table 2). The m-learning framework 
assumes that the users are on the move most of the time, with the learning taking place outside of a learning 
facility such as a classroom (Nordin et al., 2010). It allows professionals on-the-go to connect to training 
material anytime and anywhere (Sharma & Kitchens, 2005). Nedungadi and Raman (2012) suggest that m- 
learning should provide a flexible learning environment which can adapt content presentation format to suit 
the device that is being used. The delivery of information on a mobile platform needs to be conscious of the 
context of use of its users. 
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Table 2. Factors for improving the success of m-leaming projects. 


Factor 

System 

Author 

Content is instantly accessible with various 

Higher Education i.e. 

Oberer and Erkollar (2013) 

devices 

Marketing Course 

Sharma and Kitchens (2005) 

Information delivery must consider mobile 

Environmental 

Uzunboylu et al. (2009) 

environment and context of use 

Awareness 


Adaptive approaches to m-learning 

Research tool 

Nedungadi and Raman ( 2012) 

Learning should be learner-centered and allow 
for knowledge assessment 

“ 

Nordin et al. (2010) 

Adoption of a community-centered approach 

- 

Nordin et al. (2010) 

Suki and Suki ( 2007) 
Bologa and Lupu (2014) 

(well established social learning networks) 

ERP system 

Ability to download and view material and 

- 

Sharma and Kitchens (2005) 

content offline 

ERP system 

Dalveren (2014) 

Hands-on simulation of activities provided to 

Environmental 

Uzunboylu et al. (2009) 

users as they would on actual system 

Awareness 
ERP system 

SAP (2013) 


For learning to be effective Nordin et al. (2010) suggest that the learning process should be learner, 
knowledge, assessment and community centred. A community-centred approach can also be enhanced by 
social media communities (Suki & Suki, 2007). This approach is also supported by Bologa & Lupu (2014) 
who propose that well-established social learning networks can reduce the learning time of ERP systems. 

ERP m-learning applications are offered by most the larger ERP vendors. For example, SAP Education 
has developed an m-learning application, SAP Learn Now, which enables SAP users on the go, the ability to 
access the training information they need in order to maintain a competitive edge (SAP, 2013). The 
application runs on the Android and iOS operating systems and has the following features: 

• Online access to content and course material which can be downloaded and viewed offline; 

• Learners can interact with the hands-on SAP Learn Now simulation. 

• Users can receive class tests, instructor videos and have interactive learning assignments; and 

• The application enables users to highlight relevant sections, take notes and post questions on the SAP 
learning community which can be answered by subject matter experts. 

2.3 The Technology Acceptance Model (TAM) 

The Technology Acceptance Model (TAM) has been accepted by a number of researchers in an attempt to 
explain why individuals choose to adopt or not adopt a particular technology when performing a task. TAM 
is built on the theory of reasoned action (TRA) which proposes that a person’s behavioural intention initiates 
his or her performance to carry out a specified activity (Joo & Sang, 2013). TAM is based on two variables 
that influence the adoption: Perceived Usefulness (PU) and Perceived Ease of Use (PEOU). Several studies 
(Ngai, Poon, & Chan, 2007;Liu, Liao, & Pratt, 2009) on e-learning have used TAM to predict the acceptance 
of e-learning systems. These two variables can therefore be used to evaluate e-learning and m-learning 
systems (Figure 1). According to Joo and Sang (2013) PEOU refers to the degree to which an individual 
believes that using a particular system would be effortless. There are three common items which can be used 
as metrics for PEOU, namely: 1) I found the system easy to use; 2) Learning to use the system was easy for 
me; and 3) It was easy to become skilful at the system. PU is defined as the degree to which a person believes 
that using the particular technology would improve his or her job performance. Five items can be used to 
describe PU, namely: 

• The system will be of use and benefit to me; 

• The system can improve my performance (with ERP); 

• The system can help me accomplish my tasks more efficiently; 

• The system is useful for my ERP studies; and 

• The system can help me be more productive. 


40 




International Conference e-Learning 2014 


The system v.-ill he of use and benefit tome 

The system can improve my performance with ERP 

The system can help me accomplish tasks more efficiently 

The system is useful for my ERP studies 

The system can help me improve productivity 

I found the system easy to use 

Learning to use the system was easy forme 

It was easy to become skilful at the system 


Technologs- AcceptanceModel (TAXI) 
for e-learning and m-learning systems 



Perceived Usefulness (PU) 



Perceived Ease of Use (PEOU) 


Figure 1. TAM metrics for e-leaming and m-learning systems for ERP training and education programmes. 


3. RESEARCH METHODOLOGY 

The primary aim of this paper is to investigate the acceptance of e-learning and m-learning technologies 
specifically designed for ERP training and education programmes. The scope of the study will be limited to 
learners in higher education institutions in a developing country like South Africa. The research problem to 
be investigated is that whilst several ERP vendors are offering these new paradigms in learning, the 
acceptance of these technologies in developing countries is not known. Africa has been shown to have 
slightly different challenges as compared with other first world countries, since issues such as bandwidth can 
impact the success of e-learning and m-learning programmes. An improved understanding of the challenges 
involved with e-learning and m-learning for ERP systems is required in order to address the lack of quality 
ERP training programmes in both education and industry in Africa. In this way the demand for ERP skills 
and knowledge can also be met. The main research question of this paper is “ What are the success factors 
and acceptance of e-learning and m-learning technologies for ERP courses in higher education in 
developing countries? The main purpose of this study is therefore to establish the level of acceptance of e- 
learning and m-learning for teaching an introduction to ERP systems to novice ERP learners. In order to 
address the main research question, two secondary research objectives need to be realised, namely: 

• ROl: Determine the PEOU and PU of the openS AP e-learning system; and 

• R02: Determine the PEOU and PU of the SAP Learn Now m-learning system. 

The research strategies used were a literature review and two field studies in order to further investigate 
the research problem and determine the acceptance of SAP’s e-learning and m-learning systems. The two 
field studies were undertaken in and ERP systems course at the Nelson Mandela Metropolitan University 
(NMMU) in South Africa. In the first field study learners evaluated the PEOU of SAP’s openSAP e-learning 
system and the second field study the SAP Learn Now m-learning system was evaluated. The field studies 
were conducted in two practical sessions of the ERP course module. The practicals are aimed at supporting 
the weekly face to face lectures conducted by the lecturer. In these practical sessions learners get hands-on 
experience of the ERP systems. Prior to this study all learning content for this course related to the practical 
sessions was delivered in a traditional format where learners are provided with a document specifying a step- 
by-step tasks list to follow in the ERP system and then answer some questions at the end. In the first field 
study the learners were required to view the benefits offered by SAP HANA Business Suite for Sales, Service 
and Marketing. The learners were then required to complete an online assessment of the course material in 
order to evaluate their understanding of the course material. In the second field study learners made use of 
the information on the Management Accounting module using a mobile application (Table 3). 


Table 3. Summary of Field Studies. 


Field 

Study 

Learning 

System 

Outcomes/Tasks 

1 

OpenSAP 

• 

Review Sales, Services and Marketing Material 


e-learning 

• 

Complete assessment 

2 

SAP Learn 

• 

Review Performing Posting to Management Accounting 


Now m- 

• 

Complete tasks on the hands-on simulation of the posting activity 


learning 

• 

Complete assessment based on material reviewed 
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A paper-based questionnaire was compiled based on other studies (Ngai, Poon, & Chan, 2007;Liu, Liao, 
& Pratt, 2009) done on TAM for e-learning. The questionnaire was distributed to learners for completion 
after they had successfully completed all the tasks in the practical session. Since the questionnaire items have 
been used in previous studies, both the content and face validity of this study was established. Furthermore 
the questionnaire was statistically checked by an expert for reliability and validity. The questionnaire 
consisted of closed-ended questions with a 5-point Likert scale where learners had to rate the PEOU and PU 
items from 1 to 5, where 1 represents Strongly disagree and 5 represents Strongly agree. The following 
statistical ranges were applied: negative [1 to 2.6), neutral [2.6 to 3.4] and positive (3.4 to 5], In addition to 
the Likert scale questions, open ended questions were used in order for learners to express their opinions on 
the two systems being evaluated. The qualitative data was thematically analysed where themes were 
determined (Thomas, 2006). 


4. RESULTS AND DISCUSSION 

In the sections to follow the results captured from the questionnaires will be discussed in greater detail. 

4.1 Participant Profile 

The number of learners that were registered for the ERP module was 25 with the majority (64%) being male 
and the remainder were female. All of the learners are Information Systems (IS) majors and are therefore 
expert computer users but are novice ERP users since none of them had any prior experience with any ERP 
system, the SAP system or with either learning platform. Most of the learners were between the ages of 18 
and 25, with only two learners being over the age of 25. The sample size for the field studies was 23 as 
opposed to the 25 learners registered for the module. This is due to the fact that two learners did not attend 
the practical session for the openSAP practical when the questionnaires were administered. An additional 
participant was removed since it was an outlier, resulting in a final sample size (n) of 22. 

4.2 TAM Results 

The PEOU was analysed for the two systems (Table 4). After analysis of the results it can be noticed that the 
overall mean rating for PEOU for the SAP Learn Now m-learning system (p = 3.9) was only slightly higher 
than that of the openSAP rating (p = 3.6) and both of the scores were positive. From this result it can be 
deduced that the learners found the m-learning system easier to use than that of the e-learning system. From 
this information it can be deduced that the learners perceived the usefulness of the m-learning system (p = 
4.0) to be the same as that of the e-learning system (p = 4.0). Further the low standard deviation values of 
both systems reveal that the difference in opinions of the system was around the mean and therefore opinions 
did not vary that much. 


Table 4. Technology Acceptance (PEOU and PU) Results. 




PEOU 


PU 

System 

OpenSAP 

SAP Learn Now 

OpenSAP 

SAP LearnNow 

n 

22 

22 

22 

22 

Mean 

3.6 

3.9 

4 

4 

StdDev 

0.7 

1 

0.7 

1 

Min 

2 

2.3 

2.6 

2.4 

Max 

5 

5 

5 

5 


Further analysis was conducted in order to gain more insight on the specific items which consists of the 
PEOU and the variation to the mean values which were recorded for the two systems (Figure 2). The SAP 
Learn Now m-learning system was rated higher than the openSAP e-learning system for all three metrics of 
PEOU. The highest rated item (p = 4.1) was "Learning to use the e-learning/m-learning system was easy for 
me ” and was for the SAP Learn Now m-learning system. Five of the six scores were in the positive range. 
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The only metric not in the positive range was, “It was easy to become skilful at the system ” (p = 3.3), which 
was for the openSAP e-learning system. One aspect which needs to be investigated in further studies is the 
impact of the content on the ratings for the m-learning system. In order to get a more detailed perspective on 
the five individual items relating to PU further analysis was done (Figure 3). All items for both systems were 
in the positive range for PU. The ratings were very similar for all five items for both systems. The highest 
rated item was “ Can improve my performance with ERP ” (p = 4.1). The lowest rated item was for “ Helps me 
to accomplish tasks more efficiently ” (p = 3.8). 


Perceived Ease of Use (PEOU) ratings 


■ SAP Learn Now 
openSAP 


Figure 2. Perceived Ease of Use (PEOU) ratings (n = 22). 

Mean values PU 

Improve productivity' 

Useful for my'ERP studies 

Accomplish tasks more efficiently' 

Can improve mv performance with 
ERP 

System will be of use and benefit to 
me 

0.0 1.0 2.0 3.0 4.0 5.0 

Figure 3. Mean values PU. 
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i SAP Leant Now 
openSAP 


It was easy' to become skilful at the 
system 

Learning to use the system was easy 
forme 

I found the system easy to use 

0.0 1.0 2.0 3.0 4.0 5.0 



4.3 Qualitative Results 

Through the use of open-ended questions qualitative feedback was obtained from the learners by asking them 
to comment on features of the learning systems which they liked or didn’t like. A frequency count (/) was 
calculated for each theme identified in the thematic analysis and categorised into positive (Table 5) and 
negative (Table 6) features. Only the themes with the top two frequency counts for each system are listed in 
the table. The most two frequently identified features for OpenSAP were the use of videos (f = 9) and the 
quality and breadth of content (f= 9). One learner stated that “It is easier to learn from videos than from 
text Another learner highlighted the real life examples used during the video lectures as a positive feature. 
This supports the results of studies (Nedungadi & Raman, 2012; Siqueira et al., 2007) citing content quality 
and learning flexibility and adaptability as critical success factors of m-learning projects. During the SAP 
Learn Now session the use of videos was also identified frequently (f= 12) as a positive feature. The theme 
with the second highest frequency count was the hands on simulation (/ = 9). 
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Table 5. Top positive features. 


Themes (openSAP e-learning) 

Frequency 
Count (f) 

Sample comments 


Video lectures 
Flexibility and content 

9 

9 

I found it easier to learn from 
through text and slides 
Practical examples 

a video than having to read 

Themes (SAP Learn Now m- 
learning) 

Frequency 
Count (f) 

Sample comments 


Video tutorials 
Hands on simulations 

12 

9 

Explanation of the content via the videos 

Guided simulations; The touchscreen interface made 


learning easier 


The analysis of open-ended comments revealed that the most frequent negative feature of the openS AP e- 
learning system was the long duration of the video lectures (f = 8). This was followed by the amount of 
information that had to be consumed, that is information overload (f = 6). After the second session, which 
involved the SAP Learn Now m-learning system the fat finger problem and the size of the icons was the most 
frequently (f= 10) identified theme. This confirms the study of Uzunboylu et al. (2009) stating that designers 
of m-learning systems need to consider the environment and context of use. Other issues identified were 
related to content quality, for example information being too textual or too theoretical (f= 9). 


Table 6. Negative features. 


Themes (openSAP e-learning) 

Frequency 
Count (f) 

Sample comments 

Duration of videos 

8 

Videos too long; Some videos are too longfeasily lose 
concentration ) 

Information overload 

6 

“Too much theory’ to read ” 

Themes (SAP Learn Now m- 
learning) 

Frequency 
Count (f) 

Sample comments 

Fat finger and icons too small 

10 

Icons too small to click 

Content related issues 

9 

It is quite a lot of information to take in 


5. CONCLUSIONS AND RECOMMENDATIONS 

The results of the two field studies showed that both e-learning and m-learning systems for ERP education 
programmes can be successfully used to assist learners with improving their productivity and learning. In 
addition the systems are perceived by learners as easy to use. However these systems should be context 
aware just as they are in other learning environments. The m-learning system could be used to help those 
users who do not have the time to have a full experience with particular modules of ERP but wish to have a 
quick and easy understanding of ERP. Hence m-learning can be used to help those who need an introduction 
to ERP concepts and to get a simulated experience of hands-on tasks in an ERP system. The m-learning 
system should present content that is easy to understand. From the study the use of media content such as 
videos and audio tutorials could make the learning experience more efficient to the user. However the quality 
of content is an important issue for both e-learning and m-learning systems, and the fat finger problem needs 
to be taken into consideration when preparing the content to enable interactive experience of m-learning 
systems to be optimal. The study described in this paper was a preliminary study undertaken with a small 
sample size of ERP learners at one university. This study forms part of a larger study which involves the 
development of an m-learning system for ERP education and training purposes. In conclusion, the results 
obtained from the study indicate a positive acceptance of e-learning and m-learning by ERP learners. The 
study focused on the factors used in the TAM model and it was not always clear what the impact of content 
was on this acceptance. Future research is needed that investigates the impact of these factors on the 
acceptance of m-learning systems. 
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ABSTRACT 

This work starts from a systematic review about music education and self-regulation during learning processes. Then the 
paper identifies those meta-cognitive strategies that music students should adopt during their instrumental practice. The 
goal is applying such concepts in order to rethink the structure of a didactic e-book for instrumental music education. 
Thanks to the adoption of the IEEE 1599 standard, the paper outlines a model of active e-book able to improve learners’ 
performances through proper cognitive and multi-modal scaffolds. In the last section the design principles for an 
implementation will be proposed. 


KEYWORDS 

Self-regulation, music education, e-book, IEEE 1599 format. 


1. INTRODUCTION 

Music education for the young, particularly during early childhood (between the ages 3 and 8), requires a 
specific review of course curricula. An integration of multi-modal experiences based on activities such as 
moving, creating, playing, reflecting (Young, 2003) is required to train a symbolically fluent child 
(Jorgensen, 2002; Barrett, 2009). 

The learning environment should be able to represent activity-oriented musical experiences, where 
students, properly sustained by scaffold elements, are involved in a process of music 
construction/deconstruction. For example, according to Tomlinson (Tomlinson, 2013), expanding or 
scaffolding children’s early musical experiences and investigations, their engagement in the world of sound, 
their trans-modal redesign of known literature and song repertoire to communicate new meanings helps 
children establish strong, confident, vibrant, and creative identities in learning, communication, and 
performance. 

Recent research results emphasized the importance of seeing and hearing children’s perspectives, in order 
to embed children’s voices within curricular choices (Griffin, 2009). The path to the music knowledge for 
young people requires a redefinition to allow them to influence and actively build the course of their studies, 
namely to negotiate their own pathways by remaking texts. An active interaction in education should engage 
learning processes for children through complex, authentic communication (Kress, 2004). 

Children negotiate their own identities and pathways by remaking interactive texts and representations, 
catching in this way the essence of the alterations, transformations, musical arrangements and practices 
(Kress, 2013). 

New digital technologies allow to imagine new ways to design and implement educational text, as well as 
new interaction possibilities. Traditional publications should be reviewed in order to make them usable from 
different perspectives and along different pathways. In particular, a guided content customization should be 
provided. Nowadays, the exploration-driven learning environments are particularly relevant for music theory 
and musical instrument teaching. These frameworks emphasize key ideas such as an interactive and 
progressive investigation, and the development of an intuitive and creative way of thinking, as reported in 
(Jorgensen, 2002). 
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These supports, if properly conceived and redefined, could endorse self-regulatory cognitive and 
metacognitive strategies. Unfortunately, e-books today have a linear structure with a single path through 
content, and the lack of choice reduces the possibility to control the learning experience. For the new 
generation, the innovative e-book has to be flexible, adaptive and dynamic in the order and way in which 
content is studied. 


2. SELF-REGULATION AND MUSIC LEARNING 

Education technology, i.e. the research field that investigates how to teach and learn through media, denotes 
the self-regulation capability as a crucial element to be considered when defining a learning environment 
(Chang, 2005). As stated in (Jarvela and Jarvenoja, 2011) and ([Zimmerman, 2008), self-regulation is 
fundamental for a successful learning process since it helps students to create their own “method” and 
strengthen their study skills (Wolters et al., 2011), to apply the best strategies in order to reach their 
educational goals, to control their performance, and to evaluate their academic progress (de Bruin et al., 
2011 ). 

Learning, particularly when related to conceptually-rich domains (Azevedo, 2005; McMahon, 2002; Lin, 
2001), requires environments which support activities of goal self -regulation, controllable and revisable also 
in a participatory way (Schunk, 2008). Recent works, such as (Schunk and Zimmerman, 2007), (Graham and 
Harris) and (Kistner et al., 2010), demonstrated that self-regulated students obtain better results as regards 
both motor and conceptual performances. Staring at teaching and learning from the self-regulation 
perspective has great potential in music education, and in instrumental didactics in particular. 

Practising a musical instrument presents many challenges. For instance, students are required to practice 
over a long period of time, to focus on their goal, to face the threats typical of a competitive learning 
environment, to gather feedbacks in order to improve their performance, to overcome insecurity, to develop a 
psychological and behavioural toolkit against pressure and problems (Martin, 2008). 

Music education experiences a high drop-out rate caused by students’ sense of failure. Works such as 
(McPherson and Renwick, 2001) and (Pitts and Davidson, 2000) show that young music students are not 
methodical in their learning process. Like any other theoretical or practical discipline, music learning requires 
a good self-regulation attitude. Most indications come from a longitudinal research that investigated the 
evolution of 157 children practising a musical instrument between the ages 7 and 20 (McPherson and 
Renwick, 2011). Research results highlight the importance of self-regulation theory as a tool to support 
music knowledge and skill acquisition. 

In order to gain ability to play a music instrument, the young student has to learn and apply a number of 
behaviours aiming at improving his/her own performances, such as: managing and planning exercise activity, 
promptly reacting to his/her performance or to external feedbacks (e.g. when practising in an 
ensemble), modifying and adapting strategies, adequately setting the environment, asking for help when 
needed, in order to access useful resources for learning. 

Music mastering occurs when ad hoc scaffolds are provided to the learner, helping him/her to adopt self- 
regulated mechanisms to monitor and control performances (McPherson and Zimmerman, 201 1). 

With respect to other research areas about learning, only now the one related to instrumental practice is 
beginning to consider and apply the self-enhancing cycle, a key concept of the self-regulation learning 
theory. About self-regulation techniques young music students can adopt, the model by Zimmerman and 
Campillo (Zimmerman and Campillo, 2003) has been recently applied to the self-regulated problem-solving 
process. Three cyclical macro-phases are identified (McPherson and Renwick, 2011), as shown in figure 1. A 
detailed analysis of self-regulation cycle is fundamental to identify the proper scaffolds for active music e- 
books, as discussed in Section V. 
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Figure 1 . Cyclical phases of self-regulation. 


2.1 Forethought 

This phase is based on task analysis (goal setting and strategic planning) and management of self-motivation 
beliefs (self-efficacy, outcome expectation, intrinsic motivation) (McPherson, 2005). By autonomously 
defining medium- and long-term goals students can fix their own performance standards and become more 
motivated. Self-confident students are more effective in their learning efforts and do not give up when they 
have to face difficulties. Young musicians have to apply strategic learning behaviours (strategic planning) by 
identifying the most appropriate methods to accomplish a given task, without losing sight of the defined 
goals. An example of strategic planning for a musician is to stop performing warm up material from the 
printed score, and start playing the memorized text. Other strategies include hand annotations written on the 
score, or well-known techniques of sight reading (McPherson and Renwick, 2001). 

The idea of self-efficacy is fundamental for a musician (Bandura et al., 1999), and it implies the self- 
recognition of being a good instrumentalist. This opinion interacts with the outcome expectations (Graabraek 
Nielsen, 2008), thus affecting the continuation of musical studies. Self-efficacy drives students to evaluate 
their performance not only on the basis of external rewards but also for the intrinsic motivation of personal 
satisfaction (Zimmerman, 2002). 

2.2 Performance or Volitional Control 

We can recognize two processes that students can apply to improve their performance: self-control (self- 
instruction, imagery, attention focusing) and self-observation (self-recording and self-experimentation). Self- 
control processes help musicians to concentrate on their musical performance and to optimize efforts. For 
instance, they can adopt self-tasks to learn a difficult section during instrumental practice. This kind of self- 
instruction let students monitor their concentration during learning activities (Vygotsky, 1962). Other 
examples are: inner self-speech , also known as positive self-talk (e.g. “I can do this!”), that increases the 
focus on performance and alleviates performance anxiety; the creation of mental blueprints about specific 
goals; breaking up the whole piece into smaller sections, easier to be studied (Miksza, 2007). 

Self-observation highlights progresses - or their lack - in performance skills and instrumental techniques. 
In (Lehmann and Ericsson, 1997) the process has been decomposed into 3 kinds of representation: an aural 
representation of the target performance (i.e. how the piece should be played), a motor representation of the 
physical actions required, and finally the representation of current performance , constantly monitored and 
compared to the performance of reference the musician has in mind. Feedbacks act as cues prompts by 
suggesting to young students how to improve their performance. A clarifying example is switching from the 
interpretation of graphical symbols for crescendo and diminuendo to an inward feel about music expression. 

2.3 Self-reflection 

The third phase of the self-regulated learning cycle is based on self-evaluation, causal attribution, self- 
satisfaction/affect and adaptivity (Zimmerman and Campillo, 2003). Self-evaluation is one of the initial 
processes of self-reflection, and it implies a comparison between personal performance on one side and 
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performance of peers on the other. The latter is considered either a standard to satisfy or a goal to achieve. 
Self-regulated musicians typically attribute their success to causes that can be improved only through a 
greater effort. Adaptive inferences will lead to reflect on the best learning strategy, while defensive 
inferences will tend to limit the personal commitment and will lead to the abandonment of the activity. A 
high self-satisfaction level will push students towards new goals and challenges. 


3. TOWARDS ACTIVE E-BOOK FOR SELF-REGULATED 

The idea of making a school subject more engaging through computer-based technologies is not new at all. 
An example is the adoption of interactive blackboards in primary and secondary education in order to add 
multimedia and interactive contents to “traditional” ones. For instance, (Sunners et ah, 1994) describes a 
framework that integrates the blackboard model with a graphical user interface, and (Van Zeir et ah, 1998) 
provides an extensive overview of the data models and the knowledge sources that form the back-bone of an 
interactive Computer Aided Process Planning (CAPP) kernel. In the field of music education, this approach 
is particularly relevant, since the subject is strictly related to multimedia and multi-modal interfaces. In fact, 
e-books for music education have been already designed and released. Unfortunately, their typical approach 
provides a linear pathway through contents and allows a low degree of interaction: in other words, they are 
usually digital copies of traditional paper texts. Needless to say, digital technologies can introduce some 
aspects of novelty peculiar to music: 

• Hyper-links can easily provide further information about an instrument, an author, a music 
genre, etc.; 

• Scores can have a performance associated with, so that the learner can listen to a pre-recorded 
performance by either a professional player or an automated system; 

• Music notation can be presented in alternate formats, designed for visually impaired students or 
people affected by learning disabilities. 

However, music e-books available on the marketplace are far from the definition of “active book” 
mentioned in Section I. The study mode proposed by an active book is different from the reading mode , 
where the learner is able to freely read any part of a text, and solve any task or interactive part without any 
restriction. Rather, the model is characterized by the fact that the progress of reading (in this case studying) is 
handled by the textbook itself (Binas et ah, 2012). In this approach, there are many concurrent study paths 
related to the goals and levels of detail of a given subject. Any chapter, activity, exercise proposed to the 
learner is related to a previous path made of actions that can be seen as prerequisites. 

In study mode , different paths are offered to access contents by skill degree, personal interest, and goals to 
achieve. Contents are structured to provide guidance to learners, keeping the didactic framework supervised 
and designed to achieve the best results. 

Digital active books should let students adjust their cognitive as well as meta-cognitive behaviours 
(Aleven et ah, 2010; Azevedo et ah, 2011; Winne, 2011) thanks to appropriate scaffolds (e.g .feedbacks and 
prompts). As stated in (Aleven and Koedinger, 2002) and (Graesser et ah, 2000), scaffolds are functional to 
the development of specific self-regulation skills in the music field. For this reason, in Section V the subject 
will be discussed in depth. 

In scientific literature many different theories and visions are present about scaffolding. This 
heterogeneity resulted in a number of adaptive approaches and assistive technology solutions, often united by 
the concept of fixed and adaptive scaffolds (Azevedo et ah, 2004). Researchers have stressed how adaptive 
scaffolding in hypermedia environments can assist students in developing more sophisticated mental models, 
increasing declarative knowledge as well as the frequency of some self-regulated learning strategies 
(Azevedo and Hadwin, 2005). The mentioned studies have inspired the design and development of 
innovative learning systems employing the adaptive component in didactic self-regulation. 

Further studies (Kramarski and Hirsch, 2003) have provided empirical evidence that an adaptive 
scaffolding - based on feedbacks and prompts -when applied to scientific subjects supports the execution of 
regulation strategies. The importance of cueing and prompting has been confirmed by the work of (Azevedo 
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et al., 201 1), thus paving the way for new studies on adaptive scaffolding as a key element in the education of 
self-regulated learners. 

Nowadays, the scientific community is paying a great attention to the effectiveness of different kinds of 
scaffold on students’ self-regulation processes (Lajoie, 2005; Hadwin and Winne, 2001; Baylor, 2002; 
Puntambekar et al., 2003). Research results are used to revise the design of adaptive hypermedia learning 
environments. 


4. TECHNOLOGIES FOR AN ACTIVE MUSIC E-BOOK 

HTML5 is currently the latest release of HyperText Markup Language (HTML), specially designed to deliver 
rich content without the need for additional plugins. It can deliver a wide range of media contents, ranging 
from animation to graphics, from music to videos, etc. It can also be used to build complicated and rich web 
applications. HTML5 is cross-platform, in fact it has been designed to work on personal computers, tablets, 
smartphones, e-book readers and even smart TVs. Thanks to its features, HTML5 is revolutionizing not only 
the Web but also many other fields. With the release of the EPUB 3 specification (Garrish, 2011), HTML5 is 
officially a part of the EPUB standard. Consequently publishers are able to take full advantage of its feature set 
to add rich media and interactivity to their e-book content. 

Some relevant examples are described in (Kleinfeld, 2011) and (Choi et al., 2014). The integration of 
HTML5 into e-book readers allows a number of advanced and innovative applications, such as the support of 
audio/video, the use of geolocation to customize a work of fiction, the creation of colouring books, etc. 
Nevertheless, some features oriented to music education and instrumental practice require an appropriate 
format, able to represent music contents by maintaining compatibility with HTML technologies. This format 
could be IEEE 1599 (Baggi and Haus, 2009), an international standard aiming at a comprehensive description 
of music contents that has been approved by the IEEE Computer Society in 2008. IEEE 1599 adopts 
Extensible Markup Language (XML) in order to describe a music piece in all its aspects. XML is a simple, but 
very flexible text format which is playing an increasingly important role in the exchange and publishing of a 
wide variety of data on the Web and elsewhere. Since XML is one of the standards recognized by the World 
Wide Web Consortium (W3C), the choice of integrating IEEE 1599 into an e-book is coherent, as 
demonstrated in Section V. 

With respect to other music formats, the innovative contribution of this standard is providing a 
comprehensive description of music and music-related materials within a unique framework, ranging from 
symbolic score to catalogue metadata, from score images to performance audio tracks. Comprehensiveness is 
realized through a multi-layer environment where data are arranged within six layers, each corresponding to a 
different information type: general, logic, structural, notational, computer-driven performance, and audio. 

Music events are univocally identified through a common data structure known as the spine, and then 
instanced in the mentioned layers. 

Thanks to this concept, IEEE 1599 supports two synchronization modes; i) Inter-layer synchronization, 
which takes place among many descriptions of the same event in different layers, and ii) Intra-layer 
synchronization, which occurs among multiple instances of the same event within a single layer. An in-depth 
description of the key aspects of the standard is provided in (Ludovico, 2013). For further details, please refer 
to the official IEEE repository. Finally, it is worth citing the EMIPIU portal, 1 where an HTML5 viewer for 
IEEE 1599 documents is publicly available. 


5. DESIGN PRINCIPLES FOR AN ACTIVE MUSIC E-BOOK 

This section aims at reviewing the key features of the active e-book exposed in Section III in order to apply 
them to the self-regulated learning of music described in Section II through the technologies mentioned in 
Section IV. Let us go back to the definition of the three macro-phases typical of self-regulation cycle, namely 
forethought, performance or volitional control, and self-reflection (see Figure 1). For each of them, we can 
identify a number of scaffolds and consequently design a possible implementation in an active e-book. 


1 http://emipiu.di.unimi.it 
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As regards forethought, first we propose computer-assisted sight reading. This feature implies an 
automatic score follow ing algorithm with customizable tempo. 2 Tempo can be set either to improve 
performance or to add new obstacles. The IEEE 1599 format supports this feature, since both symbolic and 
graphical information are encoded in a single environment, specifically in the logic and notational layer. In 
this way, it is possible both to propose a specific printed version of the piece, page by page, and to 
reconstruct notation starting from its logical description, e.g. on a unique linearized staff system. In score 
editing software, these approaches are often called score view and scroll view respectively. Both graphical 
representations can be automatically scrolled at a given rate that the student can customize. 

Another scaffold regarding forethought implies asking meta-cognitive questions to the students in order to 
improve their comprehension of the difficulties to be faced during performance. Answering such questions 
allows relating students’ problems with their previous knowledge, as well as to redirect them to ad-hoc 
learning paths, potentially crossing different semantic axes. If the e-book page presented not only a score to 
be played, but also a number of final questions about its musical and extra-musical meaning, its key features 
(melody, rhythm, genre, etc.) and main performance obstacles, the e-book would become adaptive and active 
in suggesting a proper path to improve learning. For instance, if a student is experiencing difficulties with a 
jazz piece because of its swung notes, the interface can automatically propose a link to a page containing a 
historical performance: from the comparison between the printed score and the audio/video track, the concept 
of swing in jazz music can be intuitively acquired. From a technical point of view, this feature is 
implemented just taking full advantage of the hypertext possibilities. Please note that IEEE 1599 supports 
synchronization between score and audio/video contents, even with differently temporized performances. 

The second phase introduces a new set of scaffolds. As mentioned in Section II, jumping is one of the 
most adopted and successful techniques to practice. A piece can be subdivided into smaller parts on the base 
of music sections, performance difficulties, etc. The intra-layer and inter-layer synchronizations provided by 
IEEE 1599 support two kinds of jump. As regards the first one, after selecting a given score version and one 
of the corresponding audio/video tracks, the student can enjoy them not only in a linear way, but also 
jumping from a point to another, and without losing neither the score following nor the synchronization 
effect. This requires implementing a number of sensitive areas and controls in the interface. However, please 
note that the mapping of music events (time position, space coordinates, mutual synchronization, etc.) resides 
inside the IEEE 1599 document, and the interface simply has to implement devices to make them explicit. In 
this way, the learner can easily create jumps or loops on specific score sections. Besides, another kind of 
jumps is supported: IEEE 1599 describes within a single document many concurrent digital objects (all 
related to the same piece), as regards both graphical scores and audio/video performances. Consequently, 
during a session a student can compare different performers or different notations in real time, simply 
switching from one to another. Once again, score following and synchronization are preserved. 

Other scaffolds to be implemented are: noting and think-aloud processes, explicitly invoked by the 
system as a learning tool; peer seeking, intended here as help requests to other students; search for new 
sources, in order to get further information about a given music domain or to improve previous knowledge. 
Needless to say, also these scaffolds can be easily implemented in an active e-book, even if they could 
require network connectivity. 

Finally, as regards self-reflection we have identified another set of scaffolds fit for an active music e- 
book. Once again, peer seeking can be employed to improve the learning process, but in this case it implies a 
comparison between personal performance on one side and performance of peers on the other. Another 
possibility offered by this scaffold is cooperative learning, namely the translation of traditional classroom 
activities into academic and social learning experiences. In this case, peer seeking does not mean searching 
for a yardstick to measure personal performances against other students, but searching for other musicians of 
equal or similar skill to study and play with them. Web technologies, such as HTML, XML, and IEEE 1599 
can support both mentioned approaches. Possible consequences are gamification and reward mechanisms, 
two ways described in scientific literature to encourage self- regulated learning. Please note that, with an 
Internet connection available, this kind of music e-book could even push students towards distributed music 
performance. 


2 Tempo is the speed or pace of a given piece in terms of beats per minute (BPM). 
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Figure 2. Implementation proposal for an active music e-book. 


Probing and checking are two other scaffolds referable to self-reflection. The former implies that students 
conduct systematic analysis on their conceptual map, using both causal reasoning and computer aid to locate 
potential errors. An example is watching videos performed and commented by professional musicians, who 
can stimulate self-reflection. Thanks to IEEE 1599, these videos could be synchronized with all other music 
contents, and comments and hints could be enabled/disabled just like subtitles in movies. As regards 
checking , quizzes and automatically generated feedbacks on performance could help evaluating conceptual 
maps. 

An instance of the proposed model can be implemented by using Web technologies such as HTML5, 
Javascript and PHP. Figure 2 shows a possible interface for an active music e-book presenting some of the 
mentioned self-regulated processes. 

The music piece to perform is shown in multiple ways: through a standard score, through a color coding 
of pitches, through the keys to be pressed on different instruments (in this case, a keyboard, a guitar and a 
flute). Additional material is available, in the form of video performances and explanation texts (not shown in 
Figure 2). Standard controls allow to play, pause and stop the pre-recorded performance, and the audio track 
changes according to the currently selected instrument. Besides, the musician can choose a tempo to feel 
confident about. 


6. CONCLUSION 

Empirical evidences show that new media can help to innovate the paths of musical learning. Starting with 
the most recent studies on music education, an analysis was conducted on music students’ self-regulation 
capabilities, highlighting their importance in a learning experience. 

The design of traditional e-books in general - and music e-books in particular - has been rethought 
according to the macro-phases the self-enhancing cycle is made of: forethought, performance/volition 
control, and self-reflection. 

The transition from a reading mode to a study mode made some scaffolds emerge. These scaffolds aim at 
training self-regulated learners who will be able to improve their performance and maintain high motivation 
in their music studies. 

From a technological point of view, the implementation of scaffolds in a multi-modal, adaptive and 
dynamic interface is possible, thanks to the integration of HTML5 with IEEE 1599. The latter is an XML- 
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based format aiming at a comprehensive description of music materials in an integrated and fully 
synchronized environment. 

The educational approach presented in this work is driving the design of a working prototype. As shown 
in Figure 2, an early interface for an active music e-book has been already implemented. An early 
experimentation phase will take place next year, in the context of music courses for Italian middle school. 
Such a product follows the guidelines of the project “Scuola digitale - Editoria Digitale Scolastica” released 
by the Italian Ministry of Education, Universities and Research. If successful, this experience will be 
repeated at European level thanks to the cooperation with an international publisher. 
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ABSTRACT 

Vocabulary learning is the foundation of second language learning. Many E-learning systems have been developed to 
help learners to learn vocabulary efficiently. Most of these systems employ Ebbinghaus Forgetting Curve to make the 
review schedule for learners. However, learners are different in learning ability and the review schedule based on 
Ebbinghaus Forgetting Curve may be not fit for every learner. To solve the problem, this paper proposes the time- 
decayed user profile (TUP) to store the personalized Forgetting Curves for each learner. First, TUP is defined and then 
two algorithms, TUP Training Data Generation algorithm and TUP updating algorithm, are designed to train TUP. The 
experimental results show that the proposed time-decayed user profile can model the personalized learning characteristics 
of learners accurately. 
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Vocabulary learning, Ebbinghaus Forgetting Curve, User profile. Personalization, E-learning. 


1. INTRODUCTION 

Vocabulary learning is the foundation of second language learning (Horwitz, 1998; Laufer, et al. 2001; Read, 
2007). So many E-learning systems have been developed to help learners to learn vocabulary efficiently. In 
most of these systems, Ebbinghaus Forgetting Curve is employed to make the review schedules for learners 
(Read, 2001). 

German psychologist, H.Ebbinghaus, found that the forgetting started immediately after the learning and 
the process of forgetting is not uniform. At the beginning, the speed of forgetting is high, and then the speed 
of forgetting will reduce. Retention and forgetting are the function of time and the experimental results are 
described as Ebbinghaus Forgetting Curve (Wixted, 1997), as shown in Figure 1. In Figure 1, the abscissa 
axis denotes the elapsed time since learning, the ordinate axis denotes the retention of acquired knowledge, 
and the curve denotes the law of retention after learning. The forgetting curve graph shows that humans tend 
to halve their memory of newly learned knowledge in a matter of days or weeks unless they consciously 
review the learned materials. 

Ebbinghaus Forgetting Curve is built on many learners’ learning process, which is a general law for most 
of humans and doesn’t consider the individual characters of each learner. However, learners are different in 
learning ability because each one has different memory habits, memory modes, and memory characteristics, 
which makes their forgetting curves different (Webb, 2005; Schmitt, 2008). Therefore, the review schedule 
based on Ebbinghaus Forgetting Curve made by e-learning system may be not fit for every learner, which 
will reduce the efficiency of learning. 

To solve the above problem, this paper proposes a personalized user profile, named as time-decayed user 
profile (TUP), for vocabulary E-learning systems, which has the following characteristics: 

(1) Personalization. The individual forgetting curve for each learner will be generated, based on his 
learning process, to personalize the learning ability of different learners. 

(2) Rank vocabulary based on difficulty. The forgetting curve is also different when a learner learns 
words of different difficulty ranks. Generally, the difficult words are much more likely to be forgotten than 
easy words. 
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(3) Dynamically update. The user profile will be updated dynamically in order to catch the learning 
ability of a learner in different periods and status. 

The rest of the paper is organized as follows. Section 2 defines the proposed time-decayed user profile. 
Section 3 discusses how to generate the personalized time -decayed user profile for each learner. Some 
experiments are shown in Section 4 to validate the proposed user profile. Finally, summarizes are made in 
section 5. 


2. TIME-DECAYED USER PROFILE 

The user profile in E-learning system should record the individual characters of learners. In the learning 
process, the retention will decay as time goes on, which is focused on by the proposed user profile. Therefore, 
we name the proposed user profile as Time-decayed User Profile (TUP) 

Generally, the difficult words are much more likely to be forgotten than easy words. The user profile 
should record the abilities of a learner in learning vocabulary of different levels. Therefore how to rank the 
vocabulary for TUP should be discussed firstly. 

2.1 Rank Vocabulary based on Difficulty 

The target vocabulary for learning should be ranked into different difficulty levels. Then we can generate the 
learning tasks or build the forgetting curve for each difficulty level. Furthermore, the learning history of each 
word, such as the learning times, the time of recent learning, and so on, is used to generate the forgetting curve, 
which should be attached to each word in the vocabulary. The vocabulary model satisfying the above 
requirements is defined as following. 

Definition 1: Let V be the vocabulary and the vocabulary model with difficulty ranks for V , denoted 
by R , is defined as 

R = {R l ,R 2 ,..R i ,...,R 1 ,(R ] nR 2 n...nR ii = ®),(R 1 +R 2 +... + R n =V)}, (1) 

where n is the number of difficulty ranks in V . 

There exist many vocabulary difficulty ranks in fact. For example, in China the College English Test 
grades the vocabulary into six difficulty levels from CET1 to CET6. If V is the CET vocabulary, then n 
may be set to 6. 
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Figure 2. TUP Training Process 

In R each R i is a set of words and each word, denoted by w , is defined as 

w={r,lt,et) , (2) 

where r is the rank that the word belongs to. It is the learning times that the user had already done, et is the 
elapsed time since the latest learning on the word. 

2.2 Time-Decayed User Profile 

To personalize the characteristics of a learner in vocabulary learning process, user profile should record the 
personalized forgetting curve for each difficulty rank of vocabulary. The user profile satisfying the above 
requirements is defined as following. 

Definition 2: The time-decayed user profile, denoted by TUP, is defined as 

TUP = {R, fc v fc 2 ,..„ fc i ,...,fc n } , (3) 

where R is the vocabulary model with difficulty ranks as defined in (1) , R records the learning status of each 
word as defined in (2). fc, records the forgetting curve of the vocabulary of the i rank, which is defined as 

fc = { {et, ret ) 1 0 < et < 3 1, 0 < ret < 100} , (4) 

where et is the elapsed time since learning, ret is retention at the time et , which is calculated by 

original learning - relearning . 

ret = — 2 * 2.x 100. (5) 

original learning 

In Equation (5), original learning is the number of words in vocabulary learning task, relearning refers to 
the number of words that are forgotten and need to be reviewed, and the ret is the final score of retention at 
the time et. 

According to Ebbinghaus Forgetting Curve, when the elapsed time since learning is bigger than a month, 
the retention words will be remembered for a long time. Therefore it is enough to set the upper bound of 
et as 31 days, et can also be a decimal to record the elapsed time less than a whole day, for example, 
et = 0.5 means the elapsed time is 12 hours. 
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For a new learner, the values of fc can be initialized according to Ebbinghaus Forgetting Curve. To a 
specific elapsed time et , the corresponding ret can be initialized as 

ret = e ~ et ' r , ( 6 ) 

where r is the difficulty rank of vocabulary. Another way is to initialize the forgetting curve based on the 
average of other learners’ forgetting curves. 


3. TIME-DECAYED USER PROFILE TRAINING 


The training of TUP is the process to acquire the characteristics of a learner in vocabulary learning process. 
The acquisition is a dynamic process and the atom unit of acquisition is based on each elapsed time on each 
difficult rank. The general process of training the user profile is shown in Figure 2, which includes the 
following three steps. 


3.1 Generate Vocabulary Learning Task 


Vocabulary learning tasks are groups of words to test the mastery level of a learner. According to the 
definition of TUP, the learning task should be able to test the mastery of words belonging to different 
difficulty rank at different elapsed time. The learning task is defined as 


T = 


T T 

1 u ••• 

... T 


31 


‘,J 


T T 

± n.\ ■■■ ± n, 31 


(7) 


where each T t] is an atom learning task to test the mastery level of words belonging to difficulty rank r, at the 
elapsed time j . 

The process of generating vocabulary learning task is described in Algorithm 1 . 


Algorithm 1: TUP Training Data Generation 

Description: Generate the vocabulary learning tasks for a learner to 
collect training data for Algorithm 2 to train the time-decayed user profile 
for the learner. 

Input: R , the vocabulary with difficult ranks 

m, the number of words in each subtask. 

Output: Learning Task, T 

1. For r = 1 to n 

2. For et = 1 to 31 

3. While k < m 

4. Select a word w from R randomly 

5. If (R.w.r = r and R.w.et = et and w <£ Tij) 

6. Tjj=TijKj R.w 

7. Endif 

8. Endwhile 

9. If T,j .lengthen Then 

10. Select (m- Tj . length ) words from R, -Ty 

11. Endif 

12. Endfor 

13. Endfor 
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3.2 Get the Feedback from E-learning System 


Vocabulary E-learning system interacts with learners, performs the vocabulary testing tasks, tests the mastery 
level of each testing subtask in (7), and generates test results for the user profile updating in the next step. 

Because the design of E-learning system is not the main work of this paper, here we just give the 
definition of test results, which is denoted as 


TR = 


Kl - ^ 1,31 

tr. , 


Ka - Km 


( 8 ) 


where each tr t J is the result of learning task T,- , and the value of tr,j is calculated by (5). 


3.3 Update the User Profile 

Based on the test results of E-learning system, the forgetting curves of each difficulty rank can be generated 
and used to update the user profile, which is shown in algorithm 2. 

In algorithm 2, ooin test results means that there is no result for the test subtask, the values of fc is 
initialized according to (6). 


Algorithm 2: TUP Updating Algorithm 

Description : Generate the new forgetting curves to update the TUP 
according to the test results from E-learning system. 

Input: TUP 

Test Results, TR 
Output: TUP ( Updated ) 

1. For r = 1 to n 

2. For et = 1 to 31 

3. If ( tr r e ,= oo ) Then 

4. ret = e~ ilr 

5. Else 

6. ret = tr ret 

7. If ret>FCfcr.ret\ et -i Then 

8. ret = FC.fcr.ret\ e ,_i 

9. Endif 

10. Endif 

11. FC.fc r = FC.fc r u (et,ret) 

12. Endfor 

13. Endfor 


4. EXPERIMENTS 

Three experiments are shown in this section to validate the effectiveness of the proposed user profile. 

Data Set: In these experiments, we use College English vocabulary as the data set. College English is a 
second language course in Chinese university, which is ranked from level 1 to 6. The vocabulary is also 
divided into 6 ranks. So the difficult rank of the vocabulary in the experiments is 6. 

Experiment Participants: We select 60 none-English major students from different departments in a 
university. Among these students, 20 students are freshmen whose English level is level 1-2, another 20 
students are sophomores whose English level is level 3-4, the last 20 students are juniors whose English 
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level is 5~6. Also students are selected according to their scores in English course to ensure each group 
consists of students with different levels: good level, middle level, and low level. 

All 60 students are divided into two groups randomly: one is experimental group, the other is control 
group. Each group has 30 students. 

4.1 Experiment on evaluating the Accuracy of TUP 

Experimental Goal: 

This experiment is designed to validate the accuracy of TUP. 

The accuracy of TUP refers to the degree that the forgetting curves fit the actual situations of the learners. 
We first generate a learning/test task for each learner according to the forgetting curves in his TUP. The 
learning/test task consists of the words are just forgotten according to the forgetting curves. If the test result 
shows that the learner really forgets these words, and then the forgetting curve is accurate, otherwise it is not. 
The total accuracy of a forgetting curve is the average of each time on the curves. 

Experimental Process: 

(1) Make a study plan for duration of 31 days, the duration of learning time in each day is 30 minutes. 

(2) Select a student from the experimental group to participate in the learning process according to the 
study plan. During the learning process, the user profile of the student is trained. At last, the user 
profile has forgetting curves with the length of 31 days. 

(3) Generate test tasks for the student according to the forgetting curves in his user profile. 

(4) The student does the test and the score of each test is recorded to calculate the accuracy. 

(5) Repeat the steps from (2) to (4) for each student in the experimental group and consequently we get 
30 results. 

(6) Calculate the average accuracy of the forgetting curve of each difficult rank based on the above 30 
results 

(7) Calculate the total average accuracy of all the average values in step (6). 

Experimental Results: 

The experimental results are shown in Figure 3. In the figure each line is the accuracy of a forgetting 
curve belonging to a difficult rank. For example, R1 refers to the accuracy of the 1 st rank of the vocabulary 
(The words of College English level 1). Each point on the line is the average of the accuracy of the results of 
30 students. The total average of the accuracy is 0.852. 

4.2 Experiment on comparing the Accuracy between TUP and UP 
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Figure 3. The Accuracy of TUP 
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Figure 4. The accuracy comparison between TUP and UP (Ebbinghaus Forgetting Curve) 

Experimental Goal: 

The goal of this experiment is to compare the accuracy of the proposed TUP with the UP that uses 
Ebbinghaus Forgetting Curve. 

Experimental Process: 

The process of the experiment is the same as the first experiment. The participants of the experiment are 
the students of control group. 

Experimental Results: 

The experimental results are shown by the line UP in Figure 4, which is the average of all students of 
control group. The average result of Experiment 1 is also shown by the line TUP in Figure 4 for comparing 
with UP. 

The results show that TUP (the time-decayed user profile) does better than UP(the user profile using 
Ebbinghaus Forgetting Curve), which also means that TUP can be used to make the review plan in 
vocabulary e-learning system to improve the SLVL efficiently. 

4.3 Experiment on the Effectiveness in improving Vocabulary Learning 

Experimental Goal: 

The goal of this experiment is to compare the effectiveness between TUP and UP in improving 
vocabulary learning. 

Experimental Process: 

(1) Make a study plan for duration of 31 days and the duration of learning time in each day is 30 minutes. 

(2) All the students of the experimental group do the learning with the help of TUP. At the same time, all 
the students of the control group do the learning with the help of UP (the same as Experiment 2) 

(3) After the plan is completed, each student will be tested with the purpose of how they have mastered 
these respectively words. 

(4) Calculate the average of students from different grades (freshmen, sophomore, and junior) in each 
group (the experimental group and the control group) respectively. 

Experimental Results: 

The result shown in Figure 5 declares that the TUP is more effective in improving vocabulary learning 
compared with UP. 


61 




ISBN: 978-989-8704-08-5 © 2014 


a TUP "UP 



Freshmen sophomore junior average 


Figure 5. The effectiveness comparison between TUP and UP in improving vocabulary learning 


5. CONCLUSION 

Vocabulary learning is the foundation of second language learning, and a rich vocabulary makes the skill of 
listening, speaking, writing and oral easier to perform. Many E-learning systems have been developed to help 
leaners to learn vocabulary efficiently. In most of these systems, Ebbinghaus Forgetting Curve is employed to 
make the review schedule for learners. However, learners have difference in learning ability, so the review 
schedule based on Ebbinghaus Forgetting Curve may not be fit for every learner. To solve the problem, this 
paper proposed the time-decayed user profile (TUP) to store the personalized Forgetting Curves for each 
learner. First TUP is defined and then two algorithms, TUP Training Data Generation algorithm and TUP 
updating algorithm, are proposed to train TUP. The experimental results show that the proposed time-decayed 
user profile can model the personalized learning characteristics of learners accurately. 

The future work of this paper includes the following: 

(1) Develop an e-learning system with user-friendly interface to support larger experiments. 

(2) Validate the proposed user profile in larger experiments which means more participants and longer 
experimental period. 
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ABSTRACT 

What comes next in the field of academic e-leaming? Which e-learning trends will dominate the discourse at 
universities? Answering such questions is the basis for the adaptation of service strategies and IT-infrastructures within 
institutions of Higher Education. The present paper therefore introduces methodology and findings of a trend study in the 
field of e-teaching. The overall interest of the study was the analysis of life stages and future potentials of e-learning 
innovations. A content analysis has been conducted based on 427 scientific articles of leading German-speaking e- 
learning conferences. Thus, e-learning trends and hypes in academic teaching have been identified and characterised. The 
following paper focusses on two things: on the one hand, existing academic concepts of trend research in the field of e- 
learning will be discussed, and on the other hand, the above-mentioned study will be introduced. 1 


KEYWORDS 

Trends, E-Learning, academic teaching, Higher Education, content analysis 


1. INTRODUCTION 

E-learning is a concept that focusses on the use of digital technologies in education. E-learning innovations 
are define as technological or methodological e-learning forms which are perceived as new by potential users 
(Fischer, 2013) 

The integration of e-learning innovations is the current challenge for organisations in Higher Education in 
order to support learning, teaching, and administrative processes. Due to changed student needs, increased 
competition between organisations, different political and economic conditions, as well as new educational 
and technological approaches in Higher Education, institutions need to implement e-learning to generate 
additional educational and economic values (Seufert, 2008). But which e-learning formats can change 
academic teaching? Which formats dominate the current scientific discussions? Which approaches are close 
to a breakthrough? Which e-learning innovations have been or will be successful in Higher Education? 
Information about future challenges in the field of academic e-learning are needed for providing fitting 
conditions to teachers, like trainings, services, incentives or technologies. Therefor the present paper focusses 
on these questions. 

In the following, academic studies and reports will be introduced in order to discuss future developments 
in the field of academic e-learning. In the next step, we will describe a theoretical approach to analyse and 
evaluate the life cycle of e-learning trends based on the degree of public discussions. Based on this, insights 
will be provided into a study which helped to identify and evaluate potential e -learning-trends in academic 
teaching in German institutions of Higher Education. 


2. INSTRUMENTS FOR TREND EVALUATION 

In order to identify and characterise technical innovations which will have impacts on academic teaching, the 
Horizon Report will be introduced in the following chapter. The Horizon Report, which is published 
annually, identifies and characterises technological trends that are expected to have a great importance for the 
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various levels of education in the following years (http://www.nmc.org), with a focus on academic teaching 
and learning. With the Horizon Report experts in education and technology research evaluate the short (<1 
year), medium (2-3 years), and long-term (4-5 years) perspectives and effects of six technical innovations in 
the field of Higher Education (Bechmann, 2012). 

In the recent Horizon Report (2014), Flipped Classroom, Learning Analytics, 3D Printing, Games & 
Gamification, Quantified Self, and Virtual Assistants (see figure 1) were identified as trends in e-learning. 
However, a detailed review of the last Horizon Reports demonstrates the difficulties of forecasting. For 
example, in 2005 and 2006, as well as in 2011, 2012, and 2013, educational games were considered as a 
medium-term trend (two to three years). But for all that, the dissemination of game-based e-learning formats 
in academic teaching has not happened so far. As a short-term trend in 2009, 2010, 2011, and 2012, mobile 
applications (Mobile Apps) were considered. Also here the reality in institutions of Higher Education reveals 
another picture. However, other e-learning formats appear unexpectedly and diffuse rapidly in the field of 
academic teaching. A good example for this is Massive Open Online Courses (MOOCs). For the first time 
MOOCs were recorded in the Horizon Report in 2013 and immediately reached number one of all e-learning 
trends. In fact, many MOOCs exist in Higher Education and the number of publications and scientific events 
about them is growing rapidly (McAuley, 2013). 
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Figure 1. Trends in e-learning based on the Horizon Reports (2007 to 2013) 

As the Horizon Report shows, the life cycle of e-learning innovations is not linear. Often new e-learning 
approaches appear suddenly on the agenda and dominate the scientific discussion at universities for a limited 
time. But they then disappear abruptly from the agenda as fast as they appeared. That is why forecasting is 
difficult. For the identification of trends and the assessment of future potentials, scientific instruments and 
methods are needed. One of these instruments is the Innovation Hype Cycle by Gartner Group (See in: 
http ://w w w . gartner.com/technology/home .j sp) . 

The Hype Cycle is an analytic instrument developed and used by the IT research and advisory firm 
Gartner to represent the maturity, adoption, and social application of emerging technologies (see figure 2). 
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Figure 2. Gartner Hype Cycle (Gartner, 2014) 
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The Innovation Hype Cycle establishes a connection between the productive use of technologies and the 
public communication about it. This makes it possible to derive future potential of individual innovations by 
analyzing previous or current discourses. According the Innovation Hype Cycle the life cycle of technologies 
contains five phases. In the first phase (Technology Trigger), a technology breakthrough kicks off things. 
Based on early proof-of-concept stories, the interest of the public grows. In the second phase, early publicity 
produces a number of success stories. Growing public interest is then followed by the Peak of Inflated 
Expectations. In the next step (Trough of Disillusionment) public interest wanes because of failed 
experiments and implementations as well as the emergence of negative effects of the technology. The public 
interest grows again within the next phase. More examples of how organisations or users can benefit from the 
technology come to the fore, therefore the technology enters the Slope of Enlightenment. In the last phase, 
mainstream adoption starts and the Plateau of Productivity is reached. The technology’s market applicability 
and relevance are clear and paying off. As the above description of the Innovation Hype Cycle shows, public 
attention or discussion is relevant for the evaluation of technology life stages. 


3. EMPIRICAL STUDY 

Related to the general aim of this paper, an empirical study is now presented. The overall interest of the study 
was the analysis of life stages and future potentials of e-learning innovations. In order to make trends in e- 
learning in recent years at German universities visible, an investigation was carried out in 2014. The study 
should answer the following questions: Which e-learning formats dominate the current scientific discussions? 
Which approaches are close to a breakthrough? Which innovations of e-learning have been or will be 
successful in academic teaching? 

To answer these questions, a trend study based on a content analysis was performed (Langer, 2000). It 
was assumed that the intensity of discussion about e-learning innovations is related to their life stage - within 
the innovation process - and the degree of usage in academic teaching. This idea is based on the Innovation 
Hype Cycle, which has been described above. Therefore we analysed the content/topics of scientific 
contributions of two leading German-speaking e-learning conferences and publications: Gesellschaft ftir 
Medien in der Wissenschaft (GMW, http://www.gmw-online.de) and E-Learning-Fachtagungen der 
Gesellschaft ftir Informatik e. V. (DeLFI, http://fg-elearning.gi.de/fachgruppe-e-learning/delfi-tagung). 427 
scientific papers of both conferences resulted in the period from 2007 to 2013. Both conferences address 
scientists of German universities which apply e-learning in academic teaching. While the GMW has a strong 
focus on didactic innovations, e-learning applications tend to be discussed more from a technical perspective 
at the DeLFI. In combination both conferences demonstrate the technical and didactical potentials of e- 
learning innovations. 

The trend study is based methodically on the approach of qualitative content analysis (Mayring, 2008). 
First of all, the topics of the articles were identified, by looking for keywords, and then combined into 
categories. In this way a system of categories was derived inductively which covers the main topics of both 
conferences. All conference contributions of the GMW (n=234) and DeLFI (n=193) were sorted into the 
category system. For practical reasons, only the abstracts of the respective contributions (n = 427) were used. 
The following categories of e-learning innovations were distinguished within the study: 

Social Software (keywords: wiki, blogs, social networks, social bookmarking, peer learning 
etc.), 

E-Assessment (keywords: online exams, online assessments, tests, evaluation etc.), 

E-Portfolio (keywords: virtual artefacts, learning blogs etc.). 

Mobile Learning (keywords: mobile applications, smartphones, tablets, mobile devices etc.), 
Audio/Video (keywords: podcasts, videocasts, learning videos, lecture recordings etc.). 

Virtual Worlds (keywords: virtual environments, 3-D, virtual laboratory, simulations etc.). 
Learning Management (keywords: course management, campus management, IT-systems etc.). 
Virtual Classroom (keywords: online meetings, webinars, video conferences etc.). 

Open Content (keywords: open learning resources, copyrights, content repositories etc.), 

MOOCs (Massive Open Online Courses). 

The basic assumption of the study was that conclusions about the development potential of e-learning 
innovations in the German Higher Education can be drawn from the analysis of the scientific contributions 
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within the two selected e-learning conferences. As the Innovation Hype Cycle suggests, the degree of 
discussion delivers hints of the life stage of innovation. On that basis, it is assumed that frequently discussed 
innovations in these scientific conferences should have a high potential for academic teaching. 


4. FINDINGS 


The findings of the investigation are presented below. The figures in table 1 show how many papers of GMW 
and DeLFI conferences report about the respective e-learning innovations. The following assumptions can be 
derived from the results: 


• The detailed analysis of the frequency distribution over the seven years shows trends in the 
investigation period. The increase of frequency goes along with an increase of importance for 
academic teaching. 

• From the distribution of the frequencies within the two conferences, conclusions about the 
didactical or technical potentials of innovations can be drawn because both conferences are 
different in terms of their objective. The GMW is more oriented towards didactical issues, 
whereas the DeLFI targets increasingly technical topics. 

Table 1 . Findings of the study (numbers of articles per year) 
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In the following, the interpretation of the findings will be presented. To achieve the prognostic targets of 
a trend study, striking findings of the study will be formulated in the form of assumptions. We will not leave 
the field of the data, but also personal experiences from our daily work with teachers as well as own 
observations in the field of e-learning in German speaking universities will included in the assumptions. In 
addition, a constant comparison with the predictions of the horizon report has been done. But first, the 
classification of the life stages of all selected innovations based on the Garter Hype Cycles were made (see 
figure). 



Figure 3. Life stages of selected e-learning innovations 

Learning Management is the backbone of e-learning in Higher Education. Numerous articles about 
learning management have been presented continuously at both conferences. The decreasing degree of the 
scientific discussion should not be interpreted as a loss of importance, but rather for the productive usage of 
technologies for learning management in daily routines of academic teaching. 
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Considering the discussion about Social Software, it is striking that it is addressed much more often 
within the didactic -pedagogical-oriented GMW - in quantitative terms - (46) than within the DeLFI (27), 
since the DeLFI has a stronger focus on technical topics. It can be concluded that Social Software - and the 
associated learning activities like communication, co-operation, and prosumption - is currently considered 
primarily as a didactic innovation. 

In 2007, Virtual Worlds were identified as a medium-term e-learning trend in the Horizon Report, with 
an expected breakthrough time of 2-3 years. The euphoria was triggered by the public interest related to the 
application Second Life. But for all that, the scientific discussion of virtual worlds decreases continuously. 
This is confirmed by current Google statistics (Google trends), which capture and analyse general trends 
related to internet search queries (see figure 3). Virtual Worlds were hypes. They could not prevail at 
universities and will probably disappear from the e-learning agenda in the medium-term. 



Figure 4. Internet search queries in Google, analysed by www.google.de/trends (Keyword: “Virtuelle Welten”) 

E-Portfolios have become established firmly in the academic discourse about e-learning. However, it is 
more surprising, therefore, that e-portfolios have not been included in the internationally oriented Horizon 
Report. It cannot be determined whether e-portfolios are merely a phenomenon in the German-speaking area 
of Higher Education or not. In any case, the investigation showed that e-portfolios were significantly more in 
focus at the GMW conferences (20). Therefore they are probably more a didactical and organisational 
innovation in academic teaching than a technical challenge. 

From 2009 to 2012, Mobile Learning was number one of all e-learning trends in the Horizon Report. But 
what about the professional debate in the German Higher Education area? So far, mobile trends in e-learning 
have been rarely discussed at the investigated conferences. Until 2010 there were only two articles about this 
topic (in both conferences). However, mobile learning has moved into the focus of the GMW and the DeLFI 
during the past three years (12 papers in 2013) and now it seems to be established in the scientific discourse 
within German-speaking universities. 

MOOCs have unexpectedly become number one among all e-learning trends in the Horizon Report 2013. 
Throughout the period since 2007, there had been no indications suggesting this development. Neither the 
acronym MOOC nor the ideas behind it (open online teaching for large groups of learners) were identified as 
trends in the Horizon Reports. In 2013, the first articles about MOOCs were published in the GMW (6) and 
Delphi (2) proceedings. The fact that MOOCs belong to the e-learning trends with the largest development 
potential in academic teaching is shown by the practices of many universities. Numerous MOOCs have been 
developed in the past few months and the public debate about MOOCs is growing. The European Knowledge 
Centre for Open Education currently lists nearly 500 MOOCs of European universities 
(http://openeducationeuropa.eu/de/european_scoreboard_moocs). However, it is too early to assess the true 
potential of MOOCs. We believe that MOOCs are currently on the top of the hype (peak of inflated 
expectations), therefore it can be assumed that the scientific discussion will decrease in the following years. 


5. LIMITATIONS 

The presented findings are the result of an exploratory study. With this in mind, the described research design 
can lead to distortions or errors, which can have impacts on the validity of the findings. 

• Only abstracts of the contributions have been investigated. It is conceivable that in the complete 
articles topics have been addressed which are not referenced to in the abstract or that abstracts are 
enriched by modern (trend) terms to attract the attention of potential readers. In both cases, the 
assignment of the content into the categories has been incomplete or incorrect. 
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• For many e-learning innovations, there are no fixed technical terms in the scientific debate. The 
assignment of concepts to pre-defined categories is therefore difficult. Errors in the category 
allocation due to unclear terminology cannot be excluded. 

• We analysed the frequency of reports concerning e-learning innovations. The correlation between 
the frequency of reporting and the future potential of innovative e-learning applications has not been 
established empirically, but follows plausibility considerations (Rogers, 2003; Gartner, 2014). 

• Organisers of the investigated conferences often define the (main) topics. This influenced the 
spectrum of represented topics over all, as well as the focus of individual contributions. 

• The above-presented study was conducted at German-speaking conferences and therefore reflects 
the situation in Germany, Switzerland, and Austria. The structure and culture of Higher Education in 
all three countries is comparable. As the usage of digital media highly correlates with national 
characteristics of the higher educational system, the findings cannot be generalised. The situation of 
e-teaching might be very different in English or French-speaking areas. 


6. RESUME 

These above-mentioned limitations could affect the scientific quality of the results. The data and findings 
therefore should not be over-interpreted. Despite all the potential limitations, the data provide on the one 
hand a differentiated picture of the current debate focus of e-learning innovations in Higher Education. Some 
trends have become visible. On the other hand, the study describes a methodical approach to characterising 
the life cycle of innovations by analysing scientific material. 

Finally, we would like introduce possible method-based extensions of the presented study for future 
research: 

• The study could be extended to international conferences. 

• The prospects of individual innovations (e.g. MOOCs) could additionally be investigated by 
qualitative research methods (e.g. in the form of expert interviews). 

• The public discourse about e-learning in academic teaching could be investigated by using discourse 
analysis, to thereby bring out the role of individual actors and public institutions. 
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ABSTRACT 

The discussion on economically sustainable business models with respect to information technology is lacking in many 
aspects of proven approaches. In the following contribution the economic viability is valued based on a procedural model 
for design and evaluation of e-learning business models in the form of a case study. As a case study object a blended 
learning continuing education course is used. It is provided by a small technology-based education service provider, a 
typical representative of small and medium-sized companies (SMEs) in the german education market. The focus of the 
results discussion is the entrepreneurial attractiveness of such business models and recommendations for further studies 
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1. INTRODUCTION 

With the commencement of commercial use of the Internet there is an ongoing search for suitable business 
models and approaches according to their well-founded derivation. This is the theme of " business model " 
discussed quite controversial even in business computer science (e.g. Hess, 2012, p. 2). A fitting for this post 
definition of the business model concept, which emphasizes the tool character, provide Weiner, Renner and 
Kett (2010). Thus, a business model (elements and their relationships to each other) should contain in its 
conception the perspective of the customer/market, the service provision as well as economic relations with 
taken into account (Weiner, Renner & Kett, 2010, p.23). An overview of the state of the business model 
research can be found ibid. (2010, pp. 49), simultaneously the same is practiced at the current situation 
review: "business model research happens to be very strong on paper." (Weiner, Renner & Kett, p. 106). 
Case studies are an adequate research method as it is presented in this paper for the step in the direction of 
applied research. An interesting case study for business objects for business model analysis is that of 
technology-based training providers. An industry that is in publications on business model research very 
rarely in focus. For example, the activities of the E-Learning/Blended learning in highly cited science-based 
classification of business model types, according to Laudon and Trever (2011) are not explicitly classified. It 
is to be noted that in Germany the market of continuing education, as measured by the gross domestic 
product (approximately € 27 billion in 2003), long ago acquired a greater significance than the market of 
agricultural, forestry and fisheries (Dt. Bildungsserver, 2011 and DESTATIS, 2004). 

The heterogeneous supply structure in the continuing education sector is dominated by small and medium 
sized enterprises (SMEs), with the majority of vendors micro enterprises (up to 25 employees) as stated by 
Dietrich and Schade (2008), of the estimated 12 to 15 thousand providers in Germany (e.g. DIE, 2008 and 
Liinendonk, 2008). Just this small private training providers are facing enormous challenges with change 
processes described below in the teaching-learning culture and the educational technologies. 

The ubiquitous slogan of Lifelong Learning indicates that learning has become an ongoing task. This 
development on the one hand reflected in the demand for continuing education resist, exemplified by the 
following figures: "Distance learning or distance studying is for more and more people in Germany the 
method of their choice, if they continue their education in addition to work and family. [. . . ] About 387,000 
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people thus occupied last year, a state- approved correspondence course or a distance learning course at a 
remote high school. These are 2.0 percent more than last year. Since 2005, the ‘'‘number of participation” 
figures are even increased by 25 percent." (Forum-Distance-Learning, 2010, p. 1). On the other hand, in the 
wake of increasing demand for part-time training opportunities, the demands grow on the design of 
appropriate teaching -learning scenarios. A long time are the so called “new media” no new media anymore, 
the use of the Internet and multimedia elements is steadily increasing in the context of continuing education 
courses in Germany, from 2,821 in 2011 to 5,664 in 2013 (Dt. Bildungsserver, 2011a). To summarize it, this 
is the way get to the point: "E-learning - or more generally, the use of new media in education - is not only 
the continuation of the usual formation of new funds, but will lead to restructuring, which are difficult to 
predict today. The educational institutions will have to set up that they become permanent “construction 
areas”." (Sesink, 2006, quoted in Appelt, 2010, p. 151). 

The reaction and the implementation of the answers to the challenges outlined in the continuing education 
market should ever prove to be feasible in the economic wattle for SME educational institutions with their 
own self- understanding and financial funding (Kraft, 2010, p. 419). This means, that the (re)-modeling of 
entrepreneurial activity has to be measured by economic criteria. A possible process model is explained in 
the following section and in section 4 the results of its application in the analysis of the economic aspects are 
presented. 


2. THE ECONOMIC EVALAUTION OF BLENDED LEARNING 
BUSINESS MODELS 

As a promising approach with practical benefits, the process model for the design and evaluation of blended 
learning business models developed by Brocke, Buddendick, Gaiser and Haug (2007) is used as a template 
for the evaluation of the case study. This process model should help to subject the design of business models 
and the results of such design to continuous economic evaluation (in an iterative cycle). The core 
consideration of profit maximisation should be achieved by the triad of a market, activity and capital model 
(Brocke et ah, 2007, p 10). Figure 1 outlines the process model: the three sub-models are evaluated using 
quantitative and/or qualitative methods. The focus of the assessment in this article is on the capital model. 

The market model is used to describe the structure of the e-learning market as well as the different 
stakeholders and their roles (Grob, vom Brocke & Bensberg, 2004), with the purpose of creating an 
innovative product (product portfolio) which gains market acceptance to the extent that it allows long-term 
business success. This includes customer segmentation, the delineation of strategic business fields, an 
assessment of market attractiveness, and an industry structure and competitor analysis which enables 
providers to classify themselves as quality or cost leaders, or as niche providers (Brocke et ah, 2007). A 
quantitative assessment is undertaken by means of a deposit page that includes estimates of opportunities and 
threats. 

In the activity model (e-learning) activities (see figure 1) are described which a (training) provider carries 
out in the course of their business activity (Grob et ah, 2004). The reconciliation of their core competencies 
with the planned e-learning business model is the basis for qualitative assessment of the activity model 
(Brocke et ah, 2007). The calculation or forecast of the withdrawal page (resource consumption) is the core 
step towards quantitative assessment of the activity model (for example by means of event -driven process 
chains or process cost accounting (e.g. according to Gutbrod & Fischer, 2004). In the capital model, the 
organisational or legal form and the financing are defined for the e-learning business model. The legal 
framework is considered, the capital requirements are determined and the available or planned lending and 
investment options in the capital market are investigated. Qualitative assessment of the capital model for the 
planning period is carried out in a scenario analysis. Methods of investment appraisal, e.g. Visualization of 
Financial Implications - VoFi, are used for quantitative evaluation. 

The payments over several periods associated with the e-learning business model are compressed to target 
values and evaluated, and can be used to compare different business model alternatives as required. 
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analysis model for 
business models in 
e-learning 



Figure 1. analysis model based on v. Brocke et al. (2007, p. 10) 

The additional comments focus on the capital model. They are operationalised for practical application 
and are used to carry out a quantitative evaluation (profitability assessment) of the subject of the case study. 


3. TECHNOLOGY-ORIENTED TRAINING SERVICE PROVIDER - A 
CASE STUDY APPROACH 

The subject of the case study is a typical participant in the training market. Scientific Learning Systems (SLS, 
www.sls-saar.de) is an SME and niche provider. Its business objective provides for a three-year training 
course leading to a qualification as a state-certified radiology assistant (MTRA). The SLS itself is a spin-off 
of the Saarland University Clinics. The goal was to create a modern, customised computer-based training 
scenario. The blended learning approach was selected for the training course. To meet the requirements and 
the professional and living conditions of those interested in the training, the MTRA blended learning course 
combines attendance periods (every six to eight weeks for two to three days) sequentially with self-learning 
phases, which are supported by multimedia elements and Internet-based communication tools. 

In this way, the drawbacks of traditional distance learning courses and pure e-learning scenarios should 
be compensated, and there should be clear advantages over the attendance-based training from which the 
course was developed. The development of blended learning training began in 2002. In 2003 the first year 
started with 20 participants (the number of participants is limited to 25 per year). A learning management 
system (LMS) was developed for the blended learning scenario. The learning scripts are distributed on this 
LMS. These scripts are enriched with visualization elements such as interactive graphics, animation and 
simulations. Numerous instructional videos were produced for a variety of subjects, such as recording 
technology, and linked into the course texts. A particular focus of this teaching-learning scenario is on learner 
support during the self-learning phases. For this purpose, a so-called multimedia response system is used, 
involving a very short response time by the lecturers. Trainees receive elaborate (e.g. Narciss, 2006) audio- 
visual feedback to their questions by e-mail; for example, if they have problems while solving a physics task. 
This intensive exchange between students and lecturers during the self-learning phases enables an individual 
learning path to be developed for each learner. If components of this type are absent from a modern teaching - 
learning scenario, or if they are inadequately designed, the learners may become dissatisfied due to a lack of 
support and inadequate individual feedback or delays in providing it (Ojstersek, 2007, p 196). 

Minocha, Schroeder and Schneider (2010) even propose a clear expectation of lecturers that they should 
assume the role of providers of feedback, particularly when e-learning components and social software are 
used. However, these components cause a considerable financial outlay on the part of the training provider. 
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Despite a sort of consolidation phase for e-learning based training options that can be currently seen, the 
development costs of computer-based training (CBT) and web-based training are still very high (Meister & 
Kamin, 2010). Likewise, learner support during the self-learning/online -phase has proven to be very costly 
overall (Bremer, 2004, p 51). 

So far, the MTRA blended learning training course has been fully booked since it started 
(recommendation rate of participants is over 90 percent; Druhmann, 2013), so that the deposit page has 
shown the maximum volume, in terms of the market model. Nevertheless, this business model has not been 
imitated by other MTRA training providers (or by those offering courses for related professions such as 
pharmaceutical technical assistants - PTA). One reason could be lack of assessment or evidence of the cost- 
effectiveness of such training opportunities, and hence that of the underlying business sector. Moreover, 
established procedures are lacking. In the following a possible approach and the results of the case study 
outlined above are described. 


4. PROCESS AND INSTRUMENTS OF ECONOMIC ASSESSMENT 


Firstly, a cost structure was developed which is specially tailored to the MTRA blended learning training 
programme. This was necessary, as so far no standard cost structure has been developed on the basis of the 
various approaches in the literature. Cost-generating activities were initially identified on the basis of the 
teaching-learning processes, and the associated cost types defined - structured according to the phases of 
design, use and development of the MTRA blended learning training measures (see Figure 2). 


cash-effective costs of producing, operating and further developing the MTRA 
blended learning training course 


development costs = one-time 
costs for initial development of ... 


operating costs per year 
(student cohort) for ... 


costs of service, maintenance and 
further development per year for ... 


Rough and fine planning of the 
blended learning training course 


Preparation, post-processing and 
support of attendance events 




Development of the teaching and 
learning concept 


Subject-based and general support 
during self-study phases 




Design of the teaching and 
learning processes 


Learner exams and certification 




Design and creation of the LMS 


Support of technical components 
(server, LMS etc.) 




Design/Development of multi-media 
components 


Other operating costs 




Own development of components 
(programs, interfaces etc.) 


Administration and personnel 
management 



Purchase, adaptation and customising 
of third-party components 


Implementation, testing and 
integration of the LMS 


Integration and trialling of the entire 
learning arrangement 


Evaluation, purchase and integration 
of third-party components 


Adaptation and supplementation of 
multi-media components 


Continuous improvement/ 
Adaptation of content 


Maintenance and further 
development of technology 


Figure 2. Pragmatic cost model for determining the payment-related expenses for production, operation and development 
of the MTRA training course. Cash-effective costs of a blended learning course (own compilation based on SEIBT, 
2005, p 43; RUMBLE in comparison with 2001 cit. at GUTBROD & FISCHER, 2004, p 3) 

The withdrawal page in the case study consists of investment and current expenses (operating costs and 
cost of maintenance, care and development), each containing fixed and variable costs. 

The investment costs include the initial production costs, which are comparable to those which arise in 
the establishment of a conventional teaching-learning scenario. Work must be carried out on the rough and 
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fine planning of the training course on a project basis (initialisation, organisation and team building, project 
management and time and budget controls, etc.). This includes, for example, the creation of a suitable 
teaching and learning concept and design of the associated processes. The one-off design costs are 
determined mainly by personnel costs (salaries). The costs of preparing the learning scripts were halved, as 
they are also used in the attendance variant. 

The non-recurring technology costs include expenses for the selection, acquisition/creation, production 
and implementation of the self-developed LMS. Further cost considerations are the desired functionality (e.g. 
with or without a virtual classroom) and the degree of integration of digital media. Licence fees in the higher 
education environment are usually moderate, thanks to discounts from manufacturers or the use of open 
source software (e.g. Moodle). The purchase of appropriate hardware plays a minor role in the initial costs. 

The running costs can be broken down into operating costs and service, maintenance and further 
development costs. The operating costs are reported per calendar year. Personnel costs incurred in the 
provision of teaching make up about 60 percent of total operating costs. They include attendance events, 
support during the self-learning phases, interdisciplinary coaching and examination costs. The number of 
students and teachers who use the LMS, the nature of the technical infrastructure and software systems, and 
organisational responsibilities determine the operating costs of the LMS. A maximum of 75 students and 14 
lecturers have access to the self-developed LMS of the MTRA training course; a 20-percent position for a 
multimedia technician is added for technical support. 

The proportion of organisational administration by the technical support for the LMS is not insignificant: 
it includes user and rights management, technical support with regards to login, browser and codec problems 
and misplaced passwords. The operation of the institute's own server and associated network also leads to 
proportional personnel and hardware costs. Other operating expenses consist primarily of a lump sum for the 
use of space at the University Hospital. In 201 1 greater efforts were made to modernise the teaching -learning 
scenario (teaching and learning methods, extending the functionality of the LMS, new learning videos, 
embedding a virtual classroom). 

The investment and running costs can be summarised as follows: 


Table 1 . Selected information on the cost side of the MTRA blended learning training 
(based on Druhmann, 2013 pp. 195) 


Costs 

Year 

Amount 

Investment costs 

2002/2003 

€248.712,- 

Operating costs 

2009 

€ 71.147,- 

Costs of service, maintenance and further development 

2009 

€ 11.067,- 

“Modernisation investment” 

2011 

€ 82.172,- 


For the Visualization of Financial Implications (VoFi) calculation the investment costs are pragmatically 
divided into proportions of two thirds for the first year and one third for the second (development) year, since 
implementation activities and the launch of the training course incurred higher costs in the first year. 

The market model is not referred to in detail at this point (see Sections 1 and 3), because the customer 
target group is defined and deposits are generated in the case study solely from the course fees collected (€ 
149, - / participant and month; max. 25 participants/year). The MTRA blended learning training course has 
operated at full capacity since its launch. 

The imputed connections of the complex deposit and withdrawal structure of a training course are shown 
below in the capital model. The VoFi was chosen as a suitable model for investment accounting. The concept 
of complete financial plans differs from the “classical” methods of investment accounting mainly in the fact 
that all payments caused by the investment are shown explicitly. This allows a relatively accurate and 
transparent assessment of the relevant series of payments, a differentiation of the financing side (equity/debt, 
e.g. short-and long-term forward -interest rates, repayment terms, etc.) and the resulting financial 
consequences. In the investment time t = 0, the initial net investment is financed through the use of own 
capital and possibly an initial loan. During the observation period, n years, of the investment, payment 
surpluses (including the liquidation surplus in t = n) are set as original payments. Taking into account 
transfers, deposits and tax payments, interim financing arrangements (temporary financing) and 
reinvestments (temporary investments) need to be made which lead to annual financial balances of zero. The 
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lending/net borrowing of each period is thus by definition zero and is derived from the gap between all 
incoming and outgoing payments in consideration of the initial stock of cash. The balance of assets and loans 
(inventory balance) shows the evolution of the target value over time. Of particular importance is the target 
or final wealth at the end of the useful life or at the end of the planning horizon. The result of the VoFi 
represents the final wealth of the investor achieved by making the investment as an absolute number. 
Particularly significant for the evaluation of business models in blended learning is the VoFi total return on 
assets, which expresses the return on investment (ROI) based on a dynamic number base. 

The VoFi total return on assets shows the interest paid on capital employed over the useful life, which can 
be compared to a calculated basis for an investment decision using the average cost of capital. The VoFi 
return on total capital g GK is calculated according to the equation below (figure 3): 


/EW M [f FK 0 + Z s 

9gk “ % j EkTfk^ 


- 1 fur EW M + FK 0 + Z s > 0 


Figure 3. VoFi return on total capital g GK (Grab, vom Brocke & Bensberg, 2004, p 8) 


In order to use this equation to assess the absolute and relative benefits of the project, not only the final 
value of the investment alternative, but also the final value of the default alternative must be determined (e.g. 
using a risk-free financial asset; see Grob, 1989, pp. 73). As additional assessment information, the VoFi the 
pay-off period can be determined: this specifies the time it will take for the net initial investment to pay for 
itself, taking into account imputed interest on equity (Grob, 1989, p 92). 

In order to make a meaningful calculation for the case study, the deposits and withdrawals incurred by the 
business model of the case study were recorded over a longer period (2002 to 2011) and adopted into the 
VoFi as elements of the payment sequence (all payments in arrears at the end of each year). 

For the running costs (operating costs and service, maintenance and development) real data were 
available until 2011; for the following years an annual cost and price increase of four percent is assumed. On 
the revenue side a one-time ten percent increase in participant fees was calculated for 2011. The observation 
period for the VoFi calculation was set to 2016 (13 student cohorts); no end of the training courses, nor a sale 
with a fictitious, incalculable, return are assumed for this calculation. The observation period selected is long 
enough to ensure that the initialisation costs are not disproportionately weighted. A result, the revenue side is 
taken into account using the total real number of students in each cohort, considered several times, and also a 
sufficiently long observation of the outlay side, including a significant overhaul of the teaching -learning 
scenario. 

The final value calculation is made in accordance with the VoFi model on the last date of the period: for 
any previous period a payment surplus (receipt surplus) is offset by interim financing (reinvestment). The 
VoFi pay-off period can be specified as an additional, time-critical figure: this indicates the period in which 
the net initial investment will pay for itself, taking into account an imputed interest rate of 4.4 percent on 
equity. The VoFi total return calculation for the case study is made for the case of 100% own financing (from 
existing cash funds). Thus, the VoFi total return is equal to the VoFi return on equity. The result is a VoFi 
own or overall profitability of 5.10 percent (see table 2). The VoFi pay-off period is outside the period under 
consideration. A payback of the capital invested is therefore not to be expected within 13 years. 
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Table 2. Extract from the VoFi of the MTRA blended learning training course (based on Druhmann, 2013, p. 199) 


VOFI of MTRA Blended Learning training course - szenario „status quo" 


t„ 

t, 

2002 

*2 

2003 

t, 

2004 

t 4 

2005 

t 5 

2006 

2007 

*7 

2008 

t # 

2009 

% 

2010 

*,0 

2011 

t„ 

2012 

t,j 

2013 

t, 3 

2014 

t M 

2015 

*,s 

2016 

direct payments 

















investments 


-82.904 

-165.808 














running costs 



-24.523 

-81J15 

-82.532 

-79.649 

-81.564 

-80.032 

-78.895 

-82214 

-85.503 

-88.923 

-92.480 

-96.179 

-100.026 

-103.027 


















cash inflow 

250.000 


53.640 

98340 

89.400 

134.100 

134.100 

134.100 

134.100 

134.100 

134.100 

147.510 

147.510 

147.510 

147.510 

147.510 

net total 


-82.904 

-136.691 

16.625 

6.868 

54.451 

52.536 

54.068 

55205 

51.886 

48.597 

58.587 

55.030 

51331 

47.484 

44.483 


















equity 

250.000 
















dept capital 

0 
















interest on dept capital 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

amortisation 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

bridge financing 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

interest on bridge financing 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

reinvestment tn, 


-167.096 

-34.582 

-52.072 

-57.638 

-110.648 

-160.418 

-210.476 

-260.419 

-305.794 

-346.746 

-396.665 

-441.778 

-482.065 

-517.497 

0 

interest on reinvestment 


0 

-4.177 

-865 

-1302 

-1.441 

-2.766 

-4.010 

-5262 

-6.510 

-7.645 

-8.669 

-9.917 

-11.044 

-12.052 

-12.937 

repayment 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

517.497 

final value EW 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

549.043 

residual debt 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

annuity 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 


















EW(t) 

250.000 

167.096 

38.760 

52.937 

58.940 

112.089 

163.184 

214.486 

265.681 

312305 

354.391 

405.333 

451.695 

493.109 

529.549 

549.043 

opportunity 

250.000 

261.000 

272.484 

284.473 

296.990 

310.058 

323.700 

337.943 

352813 

368336 

384.543 

401.463 

419.127 

437.569 

456.822 

476.922 

delta EW(t) 

0 

-93.904 

-233.724 

-231.537 

-238.050 

-197.969 

-160.516 

-123.457 

-87.132 

-56.032 

-30.152 

3.870 

32.567 

55.540 

72727 

72.121 

VOFI-Pay-off-Period 


nein 

nein 

nein 

nein 

nein 

nein 

nein 

nein 

nein 

nein 

ja 

ja 

ja 

ja 

ja 


















Long-term interest debit balance 

85% 

6,5% 

6,5% 

6,5% 

6,5% 

6,5% 

6,5% 

6(5% 

85% 

6,5% 

85% 

85% 

85% 

85% 

6,5% 

85% 

Short-term interest debit balance 

8,5% 

8,5% 

8(5% 

815% 

8,5% 

8,5% 

8,5% 

85% 

85% 

8,5% 

85% 

85% 

85% 

85% 

8,5% 

85% 

Long-term interest credit balance 

4,4% 

4,4% 

4,4% 

4,4% 

4,4% 

4,4% 

4,4% 

4,4% 

4,4% 

4,4% 

4,4% 

4,4% 

4,4% 

4,4% 

4,4% 

4,4% 

Short-term interest redit balance 

2,5% 

2,5% 

2,5% 

*5% 

2,5% 

2,5% 

2,5% 

25% 

25% 

2,5% 

25% 

25% 

25% 

25% 

2,5% 

25% 

VOFFreturn on total capital 

6,2% 

















The imputed nominal and credit interest rates were defined as follows: 

• Long-term interest on debit balance: 6.5 % (range of promotional loans for SMEs: 2.45-9.1 %; 
www.genobank.de), 

• Long-term interest on credit balance: 4.4 % (average return on 10-year government bonds 2003- 
2011; www.bundesbank.de), 

• Short-term interest on debit balance: 8.5 % (assumption, as dependent inter alia on creditworthiness, 
credit rating, existing liabilities etc. ); 

• Short-term interest on credit balance: 2.5 % (average money market interest rate 2003-2011; 
www.bundesbank.de) 

The calculated VoFi return is positive and can thus seem advantageous for the project, in terms of 
financial criteria. However, the absolute figure of 5.1 percent should be compared to the returns from 
alternative investment opportunities. Thus, the absolute size of the return is slightly above the average yield 
on 10-year government bonds (4.4 percent), which considering the difference to this very low-risk investment 
alternative represents a very small premium when the risk and profit for the training provider are taken into 
account. The VoFi Pay-off-Period of over thirteen years, calculated using the opportunity interest rate, also 
seems uninviting from a business perspective. Direct comparison with other schools or providers of MTRA 
training with national accreditation is not available, as none of the other MTRA training service providers 
nationwide have so far implemented a blended learning concept. An extension of the “peer group” for a 
comparison of returns also affords no meaningful comparisons, due to the fact that the relevant financial data 
have not been published. In addition, an analysis would need the same framework conditions for the 
comparative case studies as for the case study in this article. For example, the withdrawal page would have to 
be created using the same cost structure, and the same accounting procedure would have to be used to 
calculate the measured values (e.g. VoFi). 
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5. CONCLUSION AND OUTLOOK 

Training service provision is an industry that has been faced with profound and necessary changes in its 
activities for several years. Due to ‘'‘modern” teaching-learning scenarios, marked by time- and location- 
independent learning and a high degree of multimedia and ICT use, providers are confronted by significant 
challenges to their business models, despite a growing market. In particular, economic viability is essential 
for training providers, so that the result for this case study (in the form of equity or total return) is all the 
more disappointing. Due to the full student numbers for the MTRA training course (so far), the search for 
economic optimisation can concentrate on the activity model. An area for analysis is the use of multimedia 
and blended learning elements, because this is responsible for the majority of the initialisation costs in the 
training scenario (about two thirds). Individual attention during the self-learning phases accounts for 25-30 
percent of the total ongoing costs during the provision of training: these are mainly personnel costs for 
teachers. Thus changes in the training scenario need to be justified from an economic, but also from an 
educational perspective. For this purpose it will be necessary to carry out more interdisciplinary education- 
economic cost-benefit analyses within larger samples. This is because the assessment of the economic 
viability of this blended learning business model was performed using a customised cost model and an as yet 
infrequently used investment calculation method (VoFi). Both instruments could be used in further studies. 

On the one hand this could increase the frequency of use of the application itself, so that the number of 
results achieved in the same way would be increased, which in turn would allow comparisons between 
business models. On the other hand, the cost model and the complete financial plan could be further tested 
and developed regarding their suitability for economic assessment. In addition, the approach used for this 
work could be combined with current approaches to cost -benefit analysis in further education (e.g. Schlicht, 
2012; Zangemeister, 2000), in order to systematically and comprehensively determine the cost-effectiveness 
of teaching and learning with digital media. 
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ABSTRACT 

The Ethiopian development plans have dealt with education sector as a key strategic pillar. There have been a lot of 
educational efforts. Especially to cope with lack of qualified teachers, lack of good teaching models, and remote rural 
regions separated from educational benefits, Ethiopian Ministry of Education has made efforts in utilizing educational 
media such as satellite TV program for improving quality of secondary education for last decades of years. However, 
there was lack of national investigation in the aspect of systemic evaluation, for measuring the effectiveness of satellite 
TV program. This study aims to investigate the actual practice and effectiveness of satellite TV program in Ethiopian 
secondary schools. To achieve the goal, one questionnaire was developed based on CIPP model. Two secondary schools 
are selected and 228 students (Grade 9-12) participated in the survey. Data collected from them were utilized for 
descriptives & frequencies analysis, chi-square test, and multiple regression analysis. The results indicated that Ethiopian 
students utilizing satellite TV program scored highly in the evaluation areas of context, input, process, and product of the 
program. It was also found that learning demand, learning content, and class management were factors affecting the 
satisfaction in satellite TV program. These findings suggested that satellite TV program could play an important role in 
improving the quality of in Ethiopian secondary education. As a conclusion, several educational and administrative 
strategies to improve the TV program were recommended. 


KEYWORDS 

Educational ICT, Satellite TV Program, Secondary Education, Ethiopia. 


1. INTRODUCTION 

Located in the Horn of Africa, Ethiopia has made a lot of efforts for the better national economy, especially 
poverty eradication. The main national development plans are as followings (MoFED, 2013); Agricultural- 
Development-Led Industrialization (ADLI)(2002), Sustainable Development and Poverty Reduction Program 
(SDPRP) (2002/03-2004/05), Plan for Accelerated and Sustained Development to End (PASDEP) (2005/06- 
2009/10), Growth and Transformation Plan (GTP) (2010/11-2014/15). Through the PASDEP, Ethiopian 
economic growth reached average 11 percent per annum and mobile telecom network capacity increased 
0.5M users (2005/06) to 25M users (2009/10). The number of telecom customer grew from 0.56M users 
(2004/05) to 6.5M users (2009/10). CDMA wireless network covered 90 percent of Ethiopia and 10,000km 
fiber optic cable and National Network Operation Center was established. And GTP aims to eradicate poverty 
and to reach the level of a middle-income economy as of 2020-2023. These plans are aligned with 
Millennium Development Goals (MDGs). 

The Ethiopian development plans have dealt with education sector as a key strategic pillar. For example, 
GTP has goals to achieve in general education area; Scaling up educational quality by building communities’ 
sense of ownership of educational quality by initiating integrated community mobilization, at all levels, using 
every media, digitalized (plasma based) secondary education, more Alternative Basic Education Centers. For 
supporting those plans in the educational sector, several national educational plans have been implemented; 
Education & Training Policy (ETP) (1994), General Education Quality Improvement Package (GEQIP) 
I&n, Education Sector Development Program (ESDP) 1 (1997/98-2001/02), D (2002/03-2004/05), 
111(2005/06-2009/1 0), IV (20 1 0/1 1 -20 14/1 5). 
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Due to these educational plans, the system of Ethiopian education which consists of preschool, primary 
education, secondary education, TVET, and higher education, has sharply developed. Especially a lot of 
educational indicators of general education (primary & secondary education) have been on the increase. Net 
Enrollment Ratio (NER) of primary education (Grade 1 to 8) rapidly increased from 24.9% in 1996/07 to 
85.9% in 2012/13 (Grade 1-4: 95.5%; Grade 5-8: 47.3%). ESDP IV plans to achieve 100% of NER until 
2015. But there is relative distribution of each region to the national NER (Educational Statistics Annual 
Abstract, 2013). Afar has the lowest achievement (41.5%) compared with other regions (Gambella: 98%, 
Addis Ababa: 69.4%). In case of secondary education, NER (2012/13) falls sharply (Grade 9-10: 19.4%, 
Grade 11-12: 5.3%). This may be due to delayed graduation from primary school, examination barrier and 
financial barrier. And enrollment of children with special educational needs (blind, physically & intellectually 
disabled, deaf, etc.) was 6,551. Five year (2008/09-2012/13) trend of Pupil-Teacher Ratio (PTR) for grades 9- 
12 shows that PTR has been continually reduced from 41 to 28.7. But still there is regional difference in PTR 
(Somali : 47.4%; Benishangul Gumuz: 19.9%). 

In contrast with primary education, the percentage of qualified teachers is higher in secondary education. 
Nationally, of all the secondary teachers, 91.5% are qualified for their level of secondary teaching. There is, 
however, considerable variation by region in the percentage of qualified teachers (Tigray: 96.2%; Afar: 
20.8%). 

Even though there have been a lot of educational efforts, many problems still have happened in the 
Ethiopian educational sector. Especially to cope with lack of qualified teachers, lack of good teaching models, 
and remote rural regions separated from educational benefits, Ethiopian Ministry of Education has made 
efforts in utilizing educational media such as satellite TV program for improving quality of secondary 
education. 

However, up to now, there was lack of national investigation and more systemic evaluation, for measuring 
the effectiveness of satellite TV program. Accordingly, this study aims to investigate the actual practice and 
effectiveness of satellite TV program in Ethiopian secondary schools. Research problems are as follows: 

What is the present status of satellite television program in Ethiopia? 

What are the factors which are influential in students’ satisfaction in satellite TV program? 


2. CHALLENGES OF EXISTING SATELLITE TELEVISION PROGRAM 
AND NEW OPPORTUNITY 

Ethiopian government launched satellite TV program as part of the national SchoolNet Initiative in 2004, 
which is a nationwide network of Ethiopia’s secondary schools. Today, Center for Education ICT (CEICT), 
ICT-leading organization for general education under Ministry of Education (MOE), broadcasts 2978 
television programs of 10 subjects (English, mathematics, chemistry, biology, physics, geography, civics, 
economics, technical drawing, general business) for grade 9-12. This program is broadcasted via satellite and 
secondary students in class watch the programs in plasma television for 20 minutes or so out of the 40 
minutes class ([Figure 1]). 

As of 2014, there are about 2000 secondary schools including preparatory schools (Grade 11 & 12), of 
which 1278 schools are well equipped to receive the television broadcast from the center, however only 69 % 
(893 secondary schools) among 1278 secondary schools are utilizing properly the satellite TV program. Even 
though satellite TV program has contributed in improving the quality of the secondary education in Ethiopia, 
there are several main constraints in the program. First, the basic problem is a lack of electricity power. There 
are many cases where each school is not equipped with generators for frequent blackouts. Second, there is a 
lack of maintenance and accessories for the Plasma TVs and VS AT apparatus installed in the schools. Third, 
there is still a lack of proper collaboration among CEICT, Ethio Telecom, regional educational bureaus and 
schools. And the critical problem is that the TV program is delivered one way, not considering the teacher- 
student and student-student interaction. In addition, there is a limitation in satisfying a variety of special 
education needs such as cognitive & physical disability, although the program provides the service of sign 
language for students with hearing disability. 
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Figure 1. Structure of Operating Satellite TV Program for Ethiopian Secondary Education 

Ethiopian educational ministry continues to take efforts for overcoming these shortcomings in secondary 
education ([Figure 2]). 



As one strategy, the ministry has launched an initiative to build a Local Area Network (LAN) in the first 
phase for 120 secondary schools. Each school computer laboratory will be equipped with a minimum of 80 
thin client computers throughout the country. The ministry already carried out the pilot test for its 
effectiveness. Nowadays the ministry is designing another e-learning project under the General Education 
Quality Improvement Program II (GEQUIP II ), which is supported by World Bank. As a part of the efforts 
to enhance the quality of general education, especially individual, self-directed, and self-paced learning 
environments through ICT, the MOE and CEICT has designed a program targeting 300 secondary schools 
nationwide. This program hopes to enhance the existing in-class video learning service, establish a computer 
laboratory-based learning experience managed through a private cloud service, and expand the reach of 
online-based out-of-class learning using mobile devices. An extension to this program will include the 
participation of twelve secondary schools in innovative programs to explore the use of mobile-based 
technologies for strengthening the quality of general education in emerging regions. 
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3. METHOD 


3.1 Participants 

This study aims to investigate the actual practice and effectiveness of satellite TV program in Ethiopian 
secondary schools. To achieve the goal, two secondary schools including preparatory schools are selected. A 
school is located at the urban area in Addis Ababa City and B school is located at rural area in Oromia state. 
228 students (Grade 9-12) from two regions participated in the survey. 


Table 1. Participants’ Information 



Number (%) 

Location 

Urban 

114(50) 

Rural 

114(50) 

Gender 

Male 

122 (53.5) 

Female 

106 (46.5) 

Grade 

Grade 9 

40 (21.1) 

Grade 10 

71 (31.1) 

Grade 1 1 

75 (32.9) 

Grade 12 

34 (14.9) 

Total 

228 (100) 


3.2 Instrumentation 

To investigate the actual practice and effectiveness of satellite TV program in Ethiopia, a questionnaire was 
developed. It was based on the CIPP (Context, Input, Process, Product) model, which was designed by 
Stufflebeam (1985). Based on Figure 3 below, 23 items are developed (See Appendix). They are validated 
through reviewing of 3 program experts from Center for Educational ICT (CEICT), Ministry of Education 
and 10 students from a secondary school. 



Figure 3. Structure questionnaire for student 


3.3 Procedures 

Reading official documents for literature reviews, personal interviews with related officials from CEICT, and 
searching for internet sites of Ministry of Education and Ministry of Communication Information Technology, 
Ethiopia were carried out. Based on these and CIPP model, a questionnaire was developed and validated. 
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Data gathered from secondary school students were analyzed by SPSS. Descriptives & frequencies analysis, 
chi-square test, and multiple regression analysis were performed for answering to the research questions 


4. RESULTS 

The scores from some items (a 5-point Likert-type) of the survey for evaluating the effectiveness of satellite 
TV program ranged from 3.11 to 4.12, indicating that most of the students agreed that the satellite TV 
program was satisfactory (Table 2). And students responded to other questions as follows; 41.7% of students 
selected “the program is helpful for understanding the class” as the most important motive participating in the 
satellite TV program. And as a reason why the program is interrupted during their class, they chose ‘plasma 
TV trouble’ (40.4%), lack of electricity (28.1%), and others (23.6%) in order They also responded that the 
program was helpful in the aspect of ‘more understandable in learning ’(30.3%), improving English skills 
(23.2%), rich learning materials (10.5%), and more attention to the class (10.1%) in order. For a strategy of 
improving the program, they responded ‘more interesting lesson (30.7%)’, ‘better stability of program 
transmission (13.4%), better quality of screen design (1 1.8%), and more faster maintenance service of plasma 
TV (11.8%) in order. And 38.2% of respondents answered that they never watched satellite TV program in 
their class per week. 34.1% of students chose ‘no attention of teacher’ to the question ‘why they did not 
watch the program even though it was broadcasted’. 

Table 2. Mean and Standard Deviation (SD) for Main Variables Related with Satisfaction of Satellite TV Program 


Area 

Sub- Area 

N 

Mean 

SD 

Learning 

Demand 

Learner’s level 

216 

3.96 

1.114 

Learning 

Content 

Interests 

226 

4.00 

1.170 

Customized learning 

226 

3.57 

1.404 

Appropriateness 

221 

4.06 

1.208 

Compensation of face- 
to-face class 

219 

3.61 

1.313 

Understanding 

219 

3.55 

1.282 

Usefulness 

224 

3.87 

1.349 

Learning 

Environments 

Safety 

224 

3.61 

1.490 

Interaction with TV 
program 

219 

3.52 

1.342 

Human 

Resources 

Perception of teacher 

221 

4.03 

1.232 

Knowledge of teacher 

224 

4.12 

1.157 

Class 

Management 

Interaction with TV 
teacher 

221 

3.57 

1.339 

Affirmative class 

environment 

220 

3.11 

1.384 

Satisfaction 

General satisfaction of 
TV program 

222 

3.54 

1.268 

Learning contents 

156 

3.91 

1.132 

TV teacher 

157 

3.94 

1.102 


A chi-square test was performed to determine whether the levels of satisfaction (general satisfaction, 
learning contents, TV teacher) were equal irrespective of location (urban and rural), gender (male and 
female), and grade (Grade 9-12). Only levels of general satisfaction of satellite TV program were not equal 

at the variables of location <X~=16.270, df=4, p=.003) (Table 3) and gender! X 2 = 15. 146, df=4, p=.004) (Table 
4). 
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Table 3. Chi-square test of location and general satisfaction of satellite TV program 



Strongly 

disagree 

Somewhat 

disagree 

Neutral 

Somewhat 

agree 

Strongly agree 

Total 

Urban 

Rural 

Sub-total 

11 (9.7%) 
12(11%) 
23 (10.4%) 

12(10.6%) 
12 (11%) 
24 (10.8%) 

30 (26.5%) 
12 (11%) 
42(18.9%) 

42 (37.2%) 
34 (31.2%) 
76 (34.2%) 

18 (15.9%) 
39 (35.8%) 
57 (25.7%) 

113(100%) 

109(100%) 

222(100%) 






X 2 = 16.270, df=4, p=.003 


Table 4. Chi-square test of gender and general satisfaction of satellite TV program 



Strongly 

disagree 

Somewhat 

disagree 

Neutral 

Somewhat 

agree 

Strongly agree 

Total 

Male 

Female 

Sub-total 

12 (10%) 

11 (10.8%) 
23 (10.4%) 

9 (7.57%) 
15 (14.7%) 
24 (10.8%) 

24 (20%) 
18 (17.6%) 
42(18.9%) 

33 (27.5%) 
43 (42.2%) 
76 (34.2%) 

42 (35%) 
15 (14.7%) 
57 (25.7%) 

120(100%) 

102(100%) 

222(100%) 


X 2 =15.146, df=4, p=.004 


Multiple regression was conducted to determine the accuracy of the independent variables (learning 
demand, learning content, learning environments, human resources, class management) predicting the 
dependent variable (satisfaction). Regression results in Table 5 indicate that the overall model of the five 
independent variables significantly predicts satisfaction of satellite TV program. A summary of regression 
coefficients is presented in Table 6 and indicates that only three variables, learning demand, learning content, 
and class management, significantly contributed to the model. This model accounts for 37% of the variance 
in satisfaction of satellite TV program. 


Table 5. ANOVA Table 



SS 

Df 

MS 

F 

Sig. 

Regression 

45.947 

5 

9.189 

17.026“ 

.00 

Residual 

72.325 

134 

.540 



Total 

118.272 

139 





*p<.01 


Table 6. Results of Multiple Regression 


Variable 

Unstandardized Coefficients 

Standardized 

Coefficients 

T 

P 

B 

Std. Error 

Beta 

(constant) 

.637 

.366 


1.739 

.08 

Learning demand 

.159 

.064 

.192 

2.461* 

.02 

Learning content 

.327 

.105 

.273 

3.129* 

.00 

Learning environments 

.038 

.065 

.043 

.578 

.56 

Human resources 

.095 

.080 

.096 

1.193 

.24 

Class management 

.224 

.061 

.282 

3.675* 

.00 


R 2 adj=-37, “p<.05 


5. CONCLUSION 

This study investigated the actual practice and effectiveness of satellite TV program in Ethiopian secondary 
schools. The results indicated that Ethiopian secondary school students utilizing satellite TV program scored 
highly in the evaluation areas of context, input, process, and product of the program. It was also found that 
learning demand, learning content (interests, customized learning, appropriateness, supporting face-to-face 
class, understanding, usefulness) and class management (interaction with TV teacher, affirmative class 
environment) were factors affecting satisfaction in satellite TV program. These findings suggest that satellite 
TV program can play an important role in improving the quality of Ethiopian secondary education. 
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As a conclusion of this study, several educational and administrative strategies are recommended as 
follows; First, each secondary school should be equipped with basic infrastructure such as regular electricity 
and generator in blackout. 

Second, immediate technical service should be provided by regional technicians when plasma TV has a 
trouble. 

Third, motivational instructional design should be considered for improving students’ active participation 
in satellite TV program. For a strategy of improving quality of the program, 30.7% of respondents chose 
‘more interesting lesson.’ Especially for the preparation of future satellite TV program in Ethiopian 
secondary education serving e-learning utilizing clouding systems, teachers in schools and program experts 
in Center of Educational ICT, institution leading educational ICT under the Ethiopian ministry of education, 
need to have the competences of instructional design. 

Fourth, factors such as location (urban and rural) and gender should be considered carefully when 
designing, developing, and operating satellite TV program. This is why they may effect on the satisfaction of 
the program. 

Fifth, there should be countermeasures for raising teacher’s integration of the satellite TV program to the 
class. As mentioned above, 34.1% of students responded ‘no attention of teacher’ to the question ‘why they 
did not watch the program even though it was broadcasted’. It is educator’s perceived barrier to technology to 
be overcome (Robinson, 2007). As a strategy for this, the concept of homophily (Rogers, 1995) needs to be 
applied. That is, for equal colleague teacher to introduce it or to be successful at using the technology 
increases the possibility of adoption of integrating the technology into the class. It is important to share best 
practices in utilizing satellite TV program in class. 

Sixth, leadership training of ICT for principals as well as teachers can be suggested. Diffusion of an 
innovation cannot be accomplished without teachers’ and educational administrators’ engagement and 
attention. 

And many students have complained about lack of interaction of TV teacher-student, student-student and 
student-learning contents. Although radical change is difficult due to financial and technical reasons, 
alternative instructional TV such as ‘one-way video with two-way audio may be chosen; According to a 
study (Simpson, et ah, 1993), the most successful instructional TV technologies were those allowing 
continuous two-way audio communication between classrooms with either one-way or two-way video, which 
were effective both in terms of student performance and acceptance. For Ethiopian satellite TV program 
having a shortcoming of one way delivery, using two-way audio may be good alternative. And for improving 
one way delivery of learning contents, individual, self-directed, self-paced learning environments should be 
designed and developed. 

Fortunately Ethiopian educational ministry continues to take efforts for overcoming the shortcoming in 
secondary education. Nowadays Ethiopian educational ministry is designing an e-learning project under the 
General Education Quality Improvement Program II . As a part of the efforts to enhance the quality of 
secondary education, it has designed a program targeting 300 secondary schools nationwide, which hopes to 
enhance the existing in-class video learning service, establish a computer laboratory-based learning 
experience managed through a private cloud service, and expand the reach of online-based out-of-class 
learning using mobile devices. It will be very helpful for providing satellite TV program appropriate for 
learner’s learning demands and level. Education utilizing ICT has the potential to overcome the issues of cost, 
less number of teachers, and poor quality of education as well as overcome time and distance barriers 
(McGorry, 2002). However, although the use of ICT in educational settings by itself may act as a catalyst for 
change, the change does not happen always. More instructional and administrative efforts are needed for its 
successful performance. 

More researches are needed to discover more details of strategies for improving the effectiveness of the 
satellite TV program in the context of e-learning in future and to understand why secondary teachers have 
failed to integrate satellite TV program into the class. 
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APPENDIX 


Category 

Area 

Sub-Area 

Item 


Level of 

Frequency of Use 

How often do you watch Satellite TV programs in your class per week? 


Participation 

Duration of time 

How long do you watch Satellite TV programs per one class? 

Context 

(5) 

(3) 

Reason of No use 

Why don’t you watch the program? 

Learning 
Demand (1) 

Learner’s level 

Is satellite TV program appropriate at learning level of learner? 


Perception on 
Participation 1) 

Motive of Participation 

What is your most important motive participating in the satellite TV 
program? 



Interests 

Is learning content in satellite TV program interesting? 



Customized learning 

Can you study learning contents in satellite TV program customized at 
your level of learning? 


Learning 

Content 

Appropriateness 

Does learning content in satellite TV program give rich and reliable 
information? 

Input 

(9) 

(6) 

Compensation of 
Face-to-Face class 

Does satellite TV program compensate for face-to-face class? 


Understanding 

Is learning content in satellite TV program understandable clearly? 



Usefulness 

Does satellite TV program supply enough related material and cases? 


Learning 
Environments (3) 

Safety 

Is there any problem such as disconnection of transmission? 
Why is satellite TV program interrupted during your class? 


Interaction with TV program 

Is there enough interaction with satellite TV program? 


Human Resources 

Perception of teacher 

Is the television teacher helpful for understanding learning contents? 

Process 

(4) 

(2) 

Knowledge of teacher 

Does the television teacher have enough knowledge in the subject? 

Class Management Interaction with TV teacher 

Can you interact with the television teacher? 

(2) 

Affirmative class environment Do you feel close relationship with the television teacher? 

Product 

Satisfaction^) 

Overall level of satisfaction 

Is satellite TV program generally satisfactory? 

Is learning content in satellite TV program satisfactory? 
Is television teacher satisfactory? 

(5) 

Effectiveness(l) 

Level of help 

In what aspect is the program helpful? 


Improvements( 1 ) 

Overall improvements 

What is your opinion of improvements? 
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ABSTRACT 

This paper is aimed at presenting some reflections on organisation and management of SSRI online: an e-learning 
initiative started at the University of Milan (Italy) in the academic year 2004/05 and offered to students over the last ten 
years. The initiative consisted in implementing the online version of an already existing three-year bachelor degree 
(“Laurea in Sicurezza dei Sistemi e delle Reti Informatiche - SSRI”: a Degree on Security of Computer Systems and 
Networks). 


KEYWORDS 

E-learning, online bachelor degree, online degree organisation, online degree management 


1. INTRODUCTION 

SSRI online is the e-learning version of a traditional, classroom based bachelor degree on Security of 
Computer Systems and Networks, offered at the University of Milan, Italy since ten years. A detailed 
description of the project architecture leading to implementation of SSRI online can be found in (Damiani 
2005) whereas a deep analysis of students’ population registering to the online degree is given in (Frati 
2010 ). 

In this paper, after discussing the teaching model adopted for SSRI online, we focus our attention on the 
organizational model we adopted to handle SSRI online offering, and we present the business model we used 
to finance the overall project. 


2. TEACHING MODEL OF SSRI ONLINE 
2.1 Structure of Online Lectures 

The design of the online degree has been supported by consultants from Isvor Knowledge System, an Italian 
company specialized in the production of e-learning courses. These consultants worked together with the 
university staff of CTU, the e-Learning Centre of the University of Milan, in defining the teaching model and 
the technological architecture of the project. 

As described in Damiani (2005) the structure of SSRI online can be summarized as follows: 

• each online course is structured in modules corresponding to the various topics. Each module is 
composed of didactical units, associated with the various aspects of the topic and constituted by 
different activities: lectures, exercises, tests; 

• all teaching material is available to students on the CTU web platform, which provides also forum 
discussions among students and tutors; 
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• students progresses are monitored by tracking their activities and the results of the tests associated 
with each online lecture; 

• online activities are coupled with face to face meetings with teachers for course introduction, midterm 
tests and final exam. 

Content is distributed mainly by video -lectures characterised by sequences of slides or desktop capturing, 
both synchronized with the explanation given by the teacher’s voice. It should be noted that the average 
duration of each video-lecture is about one fourth of the corresponding classroom lecture. 

The exercises proposed vary from online multiple choices, to closed answer tests, to more complex essays 
requiring tutor correction, to programming, and to networking exercises that students have to develop using a 
virtual lab (Frati, 2010) (Anisetti, 2007). 

2.2 Online Support for Students 

CTU designed and implemented the Ariel.net web platform, based on the Microsoft .net technology. This 
choice dated 2004. At those times, the Learning Management Systems offer was not too wide and, after a 
benchmarking activity, it seemed easiest and more appropriate to develop a “home-made” LMS, devoted to 
completely support online degrees or courses. The qualifying functionalities of Ariel.net allow: 

• the support of one-to-one as well as one-to-many communications, both asynchronous and 
synchronous. Besides traditional e-mail and forums, Ariel.net supplies also a private messaging 
system among students and tutors integrated into each single didactical activity (instant messaging) a 
virtual bulletin board reserved to tutors to post general interest messages, a virtual classroom support 
for synchronous meetings among students and tutors/teachers; 

• self-planning of learning activities by each student, who has a suggested learning plan, but who can 
change this plan according to her/his own needs. 

• both online streaming fruition of audio/video elements, as well as download for offline fruition; 

• the support of the exercising phases of students, tracking their advance and their results; 

• the ability to closely follow and support the individual learning process of each student, through a tool 
allowing each student to annotate her/his own instance of the online material. (Damiani, 2005: 4). 

2.3 Exams 

Taking into account the particular needs of working students (as the majority of SSRI students are) SSRI 
online has been structured around a three four-month periods calendar, allowing the student to follow a 
reduced number of courses in each teaching period. Most of the courses plan two midterm tests, used not 
only for self-evaluation purposes but also for integrating the final grade. 

Reserved exams for online students have been planned at the end of each four-month period, before the 
starting of the next one, in order to clearly separate test times from learning times. Moreover, to help working 
students, tests and exams have been organized on Friday and Saturday (Damiani, 2005: 5). 


3. ORGANISATIONAL MODEL OF SSRI ONLINE 
3.1 Production of Online Materials 

As Anderson outlines: “having comprehensive and clearly stated intended learning outcomes, as well as a 
curriculum and associated teaching approaches designed accordingly, makes the task of building the ideal 
online learning system so much easier” (Anderson, 2008:124). 

Following this statement, to implement the pedagogical design, SSRI online teachers have been supported 
by a group of instructional designers, coordinated by CTU. 

The design activity started from the identification of the learning objectives of each course, followed by a 
complete re-design to identify a new content structure based on modules and units. Each unit has been 
organised in several lectures, associated with specific activities. 
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This macro-design phase has been followed by a micro-design phase, aimed at identifying, for each 
activity, the most suited way (i.e., slide sequence and/or desktop capturing synchronized with teachers voice, 
blackboard-like effect recording teachers voice and handwriting, teacher’s video recording) to deliver the 
content. Video-based screen capturing software tools like TechSmith’s Camtasia Studio® have been selected 
to facilitate teachers work, allowing them to produce autonomously almost all video lectures without 
requiring meeting CTU staff. 

3.2 Course Tutoring 

As already mentioned, SSRi target students are full time workers who hardly cope with their “work - family 
- study” balance, so assuring them a reliable and continuous support during each step of their learning 
process has been considered a key feature to improve their chances and finally to successfully achieve a 
degree. According to these purposes the SSRI online tutoring staff has been organised referring to the 
national and international literature available in 2004 about to roles and competences of tutoring. The main 
references were: (Salmon 2000) who grouped the competencies for e-moderators into five categories: (a) 
understanding online process, (b) technical skills, (c) online communication skills, (d) content expertise, and 
(e) personal characteristics; (Rotta M. and Ranieri M. 2005) and (Rivoltella 2006) who distinguished three 
different roles for tutors: instructor, as a content expert; facilitator, charged in scaffolding students; 
moderator, as a manager of tutors-students and peer to peer discussions. 

So two SSRI different tutoring roles have been identified: 

• Course expert tutor, who acts as content facilitator (for each course, one course tutor has been selected 
for each group of 40/50 students); 

• Process tutor, who acts as e-moderator, process facilitator, adviser/counsellor. 

Duties of course expert tutors and process tutor as well as their preferred interaction media are 
summarized in Table 1. 


Table 1 . Duties and interaction media of SSRI online tutors 


Tutor 

Main duties 

Preferred interaction media 

Course expert 

• Clarify course key concepts for the students. 

• Give students sparks for reflection, 
evaluating their exercises or open essays. 

• Answer any question useful to improve the 
student subject competences. 

• Support teachers in developing the course 
contents, in managing exams and face-to- 
face meetings with students. 

• One asynchronous forum for each 
course, used to promote discussion 
about course topics and day-by- 
day peer tutorship. 

• Email messages from/to students. 

• Instant messaging system used by 
students to pose questions directly 
related to a given learning step. 

Process tutor 

• Manage a preferential channel of 
communication with students. 

• Monitor all the communication and 
relationship processes developing inside the 
learning environment. 

• Coordinate and support any organizational 
and logistic process collaborating with the 
secretarial staff and the management. 

• One asynchronous forum for the 
overall community of learners. 

• Email messages from/to students. 


3.3 Learners Community Tutoring 

The whole online community (students and course tutors) is coordinated and supervised by the process tutor. 
As (Berge 2000) points out an online moderator has to perform several roles: facilitator, manager, filter, 
expert, editor, discussion leader, marketer, and helper. (Moisey 2008: 430) also underlines that "the work of 
the counsellor in an online learning environment has three aspects. The first is to be involved in the 
development of online resources that help learners to identify and address barriers to reaching their 
educational goals. The second is to interact with the learners when an intervention is required. The third is to 
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work with other institutional staff to ensure that processes and procedures support and enhance learning” 

Process tutor has to be aware of the online community climate and of the situation of every single student, 
so she daily interacts with students using both forums and emails, implementing a “push and pull” 
communication style. She answers to any student question and - at the same time - tries to prevent their 
inquiring by contacting them directly and scaffolding them in facing each difficulty. The process tutor also 
tries to keep students engaged in the online classroom activities and to promote as much as possible peer 
collaboration. 

An analysis of the different topics discussed between students and the process tutor allows to understand 
how the areas covered by this communication process are wide: tips and tricks on course contents and exams, 
discussions about students study scheduling or about their study method, motivational counselling and 
sometimes also funny questions to relieve the pressure caused by the stressful daily routine of worker 
students. Eventually this interaction is intended to develop an empathic relationship with students, since a 
mutual trusting is essential to make the process tutor support effective. 

At the same time, the process tutor pays attention also to all the other communications being developed 
inside the learning environment - mainly the discussions in course forums - because each interaction may 
give her interesting hints about the student competence improvements and about the community feelings. 

Setting this team of tutors and cope with all its management dimensions - last but not least the 
economical one - during these first ten years has required a great deal of efforts, but the working students 
who represent our target found them really helpful. 

Student feelings and opinions have been collected year by year through different surveys about their 
whole learning experience and about each single issue (teaching materials, tutoring services, learning 
management system and so on). Looking at the quality surveys collected in several years (Frati, 2010) it is 
easy to see that students always underline the importance of the constant presence and high reliability, the 
accuracy and fair play of tutors to better organise their time/energy efforts and to achieve their educational 
goals with a complete satisfaction. Aspects emphasized as the most positive are: 

• teaching materials clarity and comprehensibility, precision and completeness, exercises availability 
and usefulness; 

• teachers and tutors willingness and support; 

• information availability and clarity about the teaching organization and the related exams. 


4. BUSINESS MODEL OF SSRI ONLINE 

In this section, we discuss the economical effort required to setup and to handle SSRI online during these 
first ten years, and the decision taken by the University regarding the fees required to students registering to 
the online version of the SSRI degree. 

For the sake of simplicity, we mainly consider direct costs (i.e. money actually spent by the University) 
without estimating indirect costs due to staff time dedicated to SSRI online and amortization of the 
technological infrastructures used for this purpose. Rationale behind this choice is the fact that both staff time 
and infrastructures amortization are costs the University has to afford in any case, regardless the specific 
activities performed by the staff and the effective usage of the infrastructures. The decision to allocate 
personnel and resources to SSRI online can thus be considered an investment, which is part of the strategic 
plans of the University, more than a mere cost. 

4.1 SSRI Online Implementation Costs 

To setup SSRI online, we had to provide during the first three years all the video -lectures of the courses 
required to complete the degree: 21 mandatory courses and 6 elective courses, for a total of 183 CFUs 
(university credits, using the ECTS scheme where a full time student is expected to earn 60 credits per 
academic year). 

As discussed in the previous sections, this required first to define a methodology for traditional courses 
restructuring, to provide guidelines for video -lectures implementation and to support involved teachers in the 
micro-design process of their restructured course. The private company consultants involved to supply 
competences not present at that time in the University operate during those first three years, for a total cost of 
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around € 350.000 including V.A.T. It should be noted that a side effect of this operation has been the 
knowledge transfer from the private company to CTU, which is now perfectly trained in supporting micro- 
design and implementation of online courses. In case of future initiatives like SSRI online, the consultancy 
cost could then be completely avoided. 

The second, main setup cost the University had to afford has been an extra salary for teachers involved in 
the video-lectures implementation, requiring a significant amount of work to be completed within strict 
deadlines (thus frequently performed during evenings and weekends). To quantify this extra salary, a survey 
of similar activities in the University of Milan as well as other Italian Universities has been conducted, 
leading to the conclusion that the amount of work required to teachers can be considered proportional to the 
number of credits of each course (i.e., to its weight): as a result, an implementation cost per credit (€ 2.000) 
has been defined as the extra salary unit for the implementation of each course, and a revision costs per credit 
(€ 200) as the extra salary unit for teaching materials revision (error corrections, self-test updating, etc.) in 
the first two years after course implementation. Using these cost units, the resulting, total cost for a course of 
e.g. 6 credits (48 lecture hours in classroom) is € 14.400 over the first three years. 

4.2 SSRI Online Management Costs 

As discussed in section 3, the main activities required to manage SSRI online are: 

• teaching support provided by a tutor expert of each course for each group of 40-50 students; 

• support to the online community of students guaranteed by the process tutor. 

Costs for the first type of tutors have been estimated in terms of hours per day necessary to monitor each 
course forum answering common questions, to reply to single students email messages, to correct open 
answer essays of students that cannot be automatically handled by the platform tools. Since these activities 
are only partially linked to the number of credits associated to each course (i.e. to its length) two cost units 
have been identified: a tutorship cost per course (6 1.500 per year) and a tutorship cost per credit (€ 250 per 
year). As a result, the total tutorship cost for a course of e.g. 6 credits followed by 80 students needs two 
tutors, each costing € 3.000 per year. 

Cost of the second type of tutorship (i.e., the process tutor) is easier to compute: it is the yearly cost for 
the University (6 36.000) of an administrative person almost fully dedicated to SSRI online. 

4.3 SSRI Online Revenues 

Since no external financial support (private or public) has been obtained by the University to offer SSRI 
online, the only revenues come from the enrolment fees paid by the students. 

In the University of Milan, the ordinary enrolment fee a student has to pay is based on a declaration of the 
total income of the student family. For a bachelor degree like SSRI online, it ranges from € 768 to € 3.639 
per year, with an average around 6 1.450 (computed on the students enrolled in SSRI online over the first ten 
years). To cover the extra costs of course and process tutorship - as well as some minor structure costs 
related to exams organised during weekends to facilitate participation of employed students - the University 
decided to apply an additional yearly fee of€ 1.500 (independent from the student family income) which has 
been reduced to € 1.200 for the first cohort of students (enrolled in academic year 2004/05) taking into 
consideration the “pioneering” work they did in using for the first time (and sometimes debugging) all the 
teaching materials. 


5. EVALUATION OF SSRI ONLINE AFTER THE FIRST TEN YEARS 
5.1 Characteristics of SSRI Online Students 

To evaluate the effectiveness of the online proposal and in particular the overlapping (if any) between the 
population of students registering to SSRI online and the population of students registering to the traditional, 
classroom version of SSRI, we took in consideration two main aspects: the age of students and their 
geographical provenience (i.e., their residence address). 
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As far as the age is concerned, SSRI freshmen (i.e., students registering to the first year of the degree) 
have been divided in three categories: 

1. 18-20 years old, where the students who just finished high schools are found; 

2. 21-28 years old, where students (partly already working, partly still searching for a job) interested to 
specialize in SSRI topics are found; 

3. more than 28 years old, where working students are found, interested to raise their skills to become 
more attractive for their companies. 

The comparison between SSRI online and traditional SSRI is given in Figure 1. It is easy to note the very 
limited overlap of the two populations: more than 50% of online students are older than 28 years, and less 
than 10% come immediately after the high schools, while almost 70% of the classroom students just finished 
their high schools ad less than 8% are older than 28 years. 



Figure 1. Age of SSRI online students vs. traditional SSRI students 


As far as the provenience of students is concerned, again three categories have been identified: 

1. students resident in Crema town and its surroundings (a small town 40 km. east of Milan, where 
classroom lectures of traditional SSRI are held); 

2. students resident outside Crema and surroundings but inside Lombardia, the 24.000 km 2 , almost ten 
million inhabitants north western Italian region where both Crema and Milan are located; 

3. students resident outside Lombardia. 

The comparison between SSRI online and traditional SSRI is given in Figure 2. It is easy to note again 
the very limited overlap of the two populations: more than 50% of online students resided abroad and 
selected the online version to follow the degree without moving from their home, while two thirds of the 
classroom students come from Crema and the surroundings area. 



Figure 2. Home residence of SSRI online students vs. traditional SSRI students 


From the above comparison, it can be concluded that the SSRI online initiative addressed a type of 
students different from the “usual” ones: these students would probably never have registered for the 
traditional classroom version of SSRI, thus the net result is a significant social service, with the improvement 
of skills of persons not interested to follow a normal degree, and a net increase of the number of University 
students. 
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5.2 Performances of SSRI Online Students 

Online students’ behaviour shows in general: 

• a slower progression in their career with respect to traditional students; as seen before, most of them 
already have a job position, thus they have to study during weekends and evenings only; 

• higher grades in exams, demonstrating the stronger motivation of online students paying a significant 
extra fee and possibly stealing time to their families to obtain the degree. 

During the first ten years, 20% of online students obtained the degree (with respect to 33% of normal 
students) but with a final average degree around 100/1 10 (vs. 95/100 of traditional students). 

5.3 SSRI Online Economical Results 

A summary of the direct costs and revenues during the first ten years of SSRI online is provided in Table 2. 

Setup costs (including all taxes) are summarized in the first two columns of Table 2. Note that in 2010 the 
Ministry of Education introduced some revisions in all Italian University degrees, requiring a further 
investment in SSRI online during year 2012 to update some of the already available online courses. 

Full costs for course tutors are show in the third column of Table 2. The decrease of these costs in the last 
years are mainly due to the following reasons: 

• progressive reduction of the number of students, common to all ICT degrees in Italy (and abroad) - 
see next section; 

• reuse of previous years forums, progressively reducing tutors effort, with a consequent reduction of 
the cost units defined at the beginning of SSRI online life. 

Full cost of the process tutorship is shown in the fourth column of Table 2, where the initial presence of 
more than one administrative person (even if part-time dedicated to SSRI online) reflected in larger expenses. 

SSRI online total revenues are obviously strictly related to the number of enrolled students, as discussed 
above. As a result, sixth and seventh columns of Table 2 show an initial increase, due to the progressive 
activation of the three years of the degree, followed by a decrease due to the reduction of freshmen. Starting 
from last year (2013) a particular attention has been given to the degree promotion through social networks 
like Facebook, which led to double the first year population (economical results will obviously appear in 
spring 2014, after final payment of students fees). 

It may be interesting to consider the graph reported in Figure 3, which plots the cumulative costs and 
revenues of SSRI online over the considered period. It is easy to note that the extra fees is almost sufficient to 
cover direct costs, while the sum of ordinary enrolment fees and extra fees practically double the expenses 
over the years. By considering that - as previously discussed - SSRI online students population has very 
limited overlapping with classroom SSRI population, leading to the conclusion that without the online 
version almost none of these students would have registered for the classroom version, we may conclude that 
SSRI online surely constitutes a profitable initiative even from the purely economic point of view. 


6. CONCLUDING REMARKS 

The paper presented an analysis of the online version of a bachelor degree in Security of Computer Systems 
and Networks, summarizing its design model, describing its organizational model and evaluating its business 
model. 

Main conclusions we may draw from this analysis are the following: 

• replicating online a traditional, classroom based degree needs a careful re-design of its courses to 
make it suitable for online students; 

• tutorship (both for supporting the learning activities in each course and for helping the student 
community) is a key factor for ensuring student success; 

• the populations of online and classroom students have very little overlapping: this means that an 
online degree supplies an important teaching service for students who cannot attend a traditional 
academic degree; 

• even without specific funding, the economical result can be definitely profitable, provided that the 
University supplies human resources especially in terms of e-learning experts. 
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Table 2. Direct costs and revenues for SSRI online implementation 


private teachers 

company extra 

consultancy salaries 


€ 132.000 € 74.855 


€ 108.000 € 284.999 


€ 108.000 € 159.907 



COSTS 



TOTAL 

COSTS 

ordinary 

enrolment 

fee 

course 

tutoring 

process 

tutoring 

€33.016 


€ 33.016 


€ 106.438 

€ 24.000 

€ 337.293 

€ 176.816 

€ 170.168 

€ 48.000 

€ 611.166 

€283.397 

€ 152.180 

€ 48.000 

€ 468.087 

€384.313 

€ 134.136 

€36.000 

€ 170.136 

€398.366 

€ 106.049 

€36.000 

€ 142.049 

€ 436.483 

€ 80.734 

€36.000 

€ 116.734 

€ 402.808 

€ 48.006 

€36.000 

€ 84.006 

€ 393.069 

€ 52.622 

€36.000 

€ 173.019 

€ 379.936 

€ 57.759 

€36.000 

€ 93.759 

€342.615 


REVENUES 

extra fee for 
online 
services 


€ 

147.600 

€ 

279.000 

€ 

334.800 

€ 

340.000 

€ 

334.500 

€ 

200.000 

€ 

102.000 

€ 

' 43.500 

€ 

'33.500 


TOTAL 

REVENUES 


€ 324.416 
€ 562.397 
€ 719.113 
€ 738.366 
€ 770.983 
€ 602.808 
€ 495.069 
€ 423.436 
€ 376.115 



Figure 3. Cumulative costs and revenues for SSRI online over the first ten years 
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ABSTRACT 

The purpose of this study is to explore variables affecting the acceptance of digital textbooks in the elementary school 
environment and provide basic information and resources to increase the intention of acceptance. Based on the above 
research puiposes. Surveys were conducted using Google Docs targeting randomly selected elementary school students 
using digital textbooks. In this study, we used the measurement instruments used in the previous study, after revising 
according to the research environment, to measure the self-efficacy in using digital textbooks, subjective norms, 
observability, perceived ease of use, perceived usefulness, and intention of acceptance. We used item parceling to avoid 
model misspecification and AMOS 20.0 for statistical analysis. The research results were as follows: first the 
observability and self-efficacy in using digital textbooks affected the perceived ease of use but subjective norms had no 
effect. On the other hand, the result that subjective norms do not affect the perceived ease of use is different from our 
initial hypothesis but is consistent with previous research by Oak (2013). Second, the results of investigating the effects 
of efficacy in using digital textbooks, subjective norms, observability, and perceived ease of use on perceived usefulness, 
self-efficacy in using digital textbooks, subjective norms, and perceived ease of use, affected perceived usefulness but 
observability did not provide significant effects. Third, the elementary school students’ perceived usefulness significantly 
affected the intention of accepting digital textbooks. The results are consistent with previous studies (Davis et al., 1989; 
Kang & Kim, 2013), suggesting that the more students utilize digital textbooks and believe that a digital textbook is 
helpful to increase learning outcomes, the more easily they are able to accept digital textbooks. 


KEYWORDS 

Digital Textbook, Technology Acceptance Model, Ease of Use, Usefulness 


1. INTRODUCTION 

Since the introduction of Apple’s iPhone and iPad in Korea in 2009, the trends of using cellular phones and 
tablet PCs based on iOS and Android have been observed across the economy, society, and culture. In 
conjunction with these trends, discussions related to the utilization of these applications are vividly 
increasing in educational environments (Kang, Song, Lee, & Ku, 2010). The Ministry of Education 
announced that they will develop digital textbooks and spent a total of $11,859 billion in 2011 with the goal 
of overcoming the limitations of current textbooks, improving current class environments, and spreading 
individualized adaptive teaching and learning methods. They selected 144 schools nationwide (12 schools in 
Seoul, 63 schools in Busan, etc.) to study the effects of digital textbooks in 2013. 

Technological innovations such as digital textbooks cannot be widespread with only the government’s 
support without technology users’ attitudes on preference and ability of using the technology being 
considered (Schwarzer & Hallum, 2008). Davis suggested the TAM (Technology Acceptance Model) to 
explain variables determining technology user’s intention of accepting new technology. The core of TAM is 
that the perceived ease of use and perceived usage of new technology affect the attitude toward innovative 
products and users’ attitudes and intentions of use (Davis, 1989). The TAM model is evaluated as a general 
model without limitations associated with particular information systems, users, research condition, or time 
and country attitudes about the use of other users’ attitudes (Lou, Luo, & Strong, 2000; Venkatesh & Davis, 
2000). Another opinion about new technology acceptance, according to the social influence model of 
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technology, is that the norms of the reference group, social environment, and personal characteristics affect a 
user’s new technology acceptance (Fulk, Steinfield, Schmitz, & Power, 1987). 

However, there are constraints in that most previous studies consider only single factors rather than 
considering personal characteristics and environmental variables within a research frame. Such a frame 
would include interface design, ease of use, and the perceived usefulness (Hong, Hwang, Hsu, Wong, & 
Chen, 2011), as well as self-efficacy (Venkatesh, 2000). Thus, the current study scrutinizes the causal 
relationships between variables of personal characteristics, such as self-efficacy, subjective norms of 
environmental variables, and observable attributes of innovations by countering the above limitations. 

Bandura’s self-efficacy concept has been applied to new technology and equipment, such as computer 
self-efficacy (Albion & Ertmer, 2002; Oliver & Shapiro, 1993) and mobile self-efficacy (Kim & Kim, 2011). 
People with high self-efficacy are more likely to succeed in projects (Oliver & Shapiro, 1993). Self-efficacy 
refers to one’s own belief in being able to complete a given project successfully (Bandura, 1982) and The 
current study will investigate the research regarding students’ self-efficacy on using digital textbooks. In 
addition, we will refer to self-efficacy in using digital textbooks as a personal characteristic variable affecting 
the intention of acceptance, and subjective norms as an environmental characteristic variable (Fishbein & 
Ajzen, 1975). This environmental variable has been shown to have an important effect on technology 
acceptance (Seo, 2011). 

The last variable that we incorporated in this study is observability. Rogers (1983) studied individual 
differences in the speed of new technology acceptance. As a result, Rogers discovered that five perceived 
innovative traits significantly affect new technology acceptance. The five traits are relative advantage, 
suitability, complexity, testability, and observability (Rogers, 1983). However, previous research revealed 
that relative merits and suitability are similar to perceived usefulness, and the concept of ease-of-use and that 
of ease of use are the same concept (Lee, Hsieh, & Hsu, 2011). 

In this study, we included observability but excluded testability, given that the reliability in measuring 
testability would be compromised as the subjects had already been using digital textbooks for the semester. 
Thus far, although numerous previous studies have investigated the variables affecting the receiver’s 
intention to accept innovative techniques based on the TAM model, these studies were conducted based 
largely on e-learning and mobile environments (Lee, 2012; Gong, Xu, & Yu, 2004; Leung & Wei, 2000). 
However, research reflecting educational environments is rare. 

The purpose of this study is to explore variables affecting the acceptance of digital textbooks in the 
elementary school environment and provide basic information and resources to increase the intention of 
acceptance. Based on the above research purposes, we established the following research questions. 

Research Question 1: Does self-efficacy in using digital textbooks, subjective norms, and observability 
affect the perceived ease of use? 

Research Question 2: Does self-efficacy in using digital textbooks, subjective norms, observability, and 
perceived ease of use affect the perceived usefulness? 

Research Question 3: Does perceived ease of use and perceived usefulness affect the degree of 
acceptance? The hypothetical research model based on above hypotheses are as follows in Figure 1 as 
follows: 



Perceived 

Usefulness 


Perceived 
Ease of Use 


> Intention 
to Use 


Figure 1 . Hypothetical Research Model 
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2. RESEARCH METHODS 

2.1 Subjects 

Aside from seven incomplete survey responses, 319 out of 326 survey responses were analyzed. Out of 319, 
160 subjects were female (50.19%) and 150 were male (49.84%). There were nine third graders (2.82%), 131 
fourth graders (41.07%), 127 fifth graders (39.81%), and 52 sixth graders (16.30%) respectively. Among the 
schools selected for research, the K, O, P, and W Elementary Schools started using digital textbooks in 
March 2012, and Y and G started in March 2013. Depending upon the principal’s preference, two schools 
used digital textbooks in Korean and English subjects and five schools used digital textbooks in social studies 
and science subjects by using tablet PCs, Netbooks, and Notebook Computers. 

2.2 Research Procedures 

In this study, we conducted online surveys using Google Docs targeting randomly selected students using 
digital textbooks to explore the variables affecting elementary students’ intention of accepting digital 
textbooks, from November 25, 2013 to December 24, 2013. Subjects self-reported based on their own 
experience of using digital textbooks. 

2.3 Measurement Instrument 

In this study, we used the measurement instruments used in the previous study, after revising according to the 
research environment, to measure the self-efficacy in using digital textbooks, subjective norms, observability, 
perceived ease of use, perceived usefulness, and intention of acceptance. All items used a 5-point Likert scale 
(1 point: not at all, 2 points: no, 3 points: usually, 4 points: yes, 5 points: very much so). 

To measure self-efficacy, subjective norms, and intention for acceptance, we revised Taylor and Todd’s 
instrument to measure information technology usage. The TAM (Technology Acceptance Model) has the 
merit of combining the theory of planned behavior and the decomposed theory of planned behavior. It also 
reflects real life (1995). The measurement includes six items assessing self-efficacy in using digital textbooks 
(e.g., I feel comfortable when using the digital textbook), nine items assessing subjective norms (e.g., people 
who care about me think that I need to use digital textbooks), and three items assessing degree of acceptance 
(e.g., I would like to utilize the digital textbook for further studies). The internal consistency of the original 
instrument was Cronbach's a = .84, for subjective norms a = .80, and for intention for acceptance a = .93. 
The reliability of the current study data was .93, .91, and .91, respectively. 

To measure observability, we revised the instrument by Moore and Benbasat (1991), which measures the 
level of accepting innovative techniques in the school environment (e.g., I have seen that other friends study 
with digital textbooks). Since the instrument by Moore and Benbasat reflects the school environment, it was 
consistent with our current research environment. The reliability was Cronbach’s a = .88. Cronbach’s a for 
the four items measuring observability was .76. 

The perceived usefulness and perceived ease of use were revised and supplemented based on the 
instrument developed by Davis (1989); the reliability of this instrument has been demonstrated in many 
research studies (Agarwal et ah, 1997; Chau, 1986). The perceived ease of use was measured by four items 
(e.g., learning with digital textbooks is simple but difficult.). The perceived ease of use was measured by four 
items (e.g., I believe that I can study better with digital textbooks). The reliability of the original instrument is 
Cronbach’s a = .94 and .91. The Cronbach's a for the current study was .93 and .86, respectively. 

2.4 Data Analysis Method 

We established a statistical model to investigate the causal relationships among self-efficacy in using digital 
textbooks, subjective norms, observability, perceived ease of use, and perceived usefulness based on a 
hypothetical research model and tried to measure the latent variables by using indicator measurement 
variables. Since all variables were shown to be single factors in exploratory factor analysis, we used item 
parceling to avoid model misspecification. We used AMOS 20.0 for statistical analysis. 
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3. RESULTS 

3.1 Inter-relational Matrix and Descriptive Statistics 

If each measured variable in structural equation modeling does not have a normal distribution, we would get 
biased estimates and an inaccurate model. Therefore, to confirm normality in the multivariate distribution, 
mean, standard deviation, skewness, and kurtosis were examined. 

The average of values ranged from 3.11 to 3.89, standard deviation from .81 to 1.01, skewness from .11 
to .62, and kurtosis from .02 to .30 respectively. If the skewness of the variables is less than 3 and kurtosis is 
less than 10, a normal distribution in the model can be assumed (Kline, 201 1). 

We examined the VIF (Variance Inflation Factor) to assess multicollinearity between variables and we 
confirmed that there were no problematic influencing variables, with VIF from 1.58 to 3.91 (which is 
required to be less than 10), and we confirmed significant correlations among the variables at the level of .05. 
Clearly indicate advantages, limitations and possible applications. 

3.2 Estimate of Measurement Model 

We estimated the fit of the measurement variables based on the maximum likelihood method, as a second 
confirmatory process of goodness of fit (Kline, 2011). As displayed in Table 1, the index of TLI and CFI was 
.995 and .995 respectively, which satisfies the acceptable criteria. The RMSEA value was .038, which 
suggested goodness of model fit. 


Table 1 . Fit Indices of the Measurement Model (n = 3 19) 



CMIN 

P 

df 

TLI 

CFI 

RMSEA (90% Confidence Interval) 

Measurement 

Model 

57.089 

.031 

39 

.991 

.995 

.038 (. 012 - . 058 ) 

Criteria 




>.90 

>.90 

<.08 


The results of confirmatory factor analysis provide the evidence of convergent and construct validity of 
the theoretical construct. When the correlation between the measured variables and the latent variable is 
greater than .50, convergent validity can be confirmed. When the correlation between the latent variable is 
less than .80, construct validity can be confirmed (Moon, 2009). As a result of examining the relationships 
between the latent and index variables, the standard loading factor indices of the latent variables ranged from 
.76 to .96, which were significant at the a .05. This suggests that the selected index variables to measure the 
theoretical variables in our model demonstrate convergent validity. In addition, cross -correlation between 
latent variables ranged from .28 to .51, which has discriminatory validity. Accordingly, it was shown that all 
of the latent variables of the research model have been accurately and reliably measured. 

3.3 Examination of Structural Model 

Since the model estimate was theoretically confirmed and the model’s goodness of fit satisfies the criteria, 
the fit of the regression model was estimated. The estimated results of the structural model are as follows in 
Table 2. 


Table 2. Examination of Fit of the Initial Structural Model (n = 319) 



CMIN 

P 

df 

TLI 

CFI 

RMSEA (90% Confidence Interval) 

Initial Structural 
Model 

126.515 

.000 

42 

.960 

.975 

. 080 (. 064 ~ . 096 ) 

Criteria 




>.90 

>.90 

<.08 
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The fit indices for the initial structural model were as follows: TLI = .960, CFI = .975, and RMSEA = .80. 
These data suggest a good fit with the estimated model. This means that the intention of acceptance, self- 
efficacy in using digital textbooks, subjective norms, observability, perceived ease of use, and perceived 
usefulness have statistically significant causal relationships. 

Accordingly, the results of examining the influence between variables are as follows. The effects of self- 
efficacy in using digital textbooks on perceived ease of use was P = .847 (t = 7.738, p < .05), that of 

subjective norms on perceived ease of use was P = -.123 (t = -1.446, p > .05), and that of observability on 
perceived ease of use was P = .126 (t = 2.140, p < .05). 

The effects of self-efficacy in using digital textbooks on perceived ease of use was P = .504 (t = 4.210, p 
< .05), that of subjective norms on perceived ease of use was P = .228 (t = 2.739 p < .05), and that of 
observability on perceived ease of use was P — . 1 74 (t = 2.179, p < .05). The effect of perceived ease of use 
on intention of acceptance was P = .260 (t = 3.731, p < .05) and that of perceived usefulness on intention of 
acceptance was P = .596 (t = 8.596, p < .05). 

However, since the relationships between subjective norms and perceived ease of use, and observability 
and perceived usefulness in the initial structure model were not statistically significant, we established a 
succinct revised model after removing those variables from the initial model, under the condition that there 
were no statistical differences after removing them. 

We conducted a '£ test to confirm that there was no statistical difference between the initial structural 
model and the revised model since there was a hierarchical relationship between the initial structural model 
and the revised structural model. We selected the more succinct revised model as our final research model 
since there was no statistical difference (A% 2 = 2.885, p = .236). Using maximum likelihood analysis, the 
revised structural model showed a good fit with TLI = .962, CFI = .975, RMSEA = .078 as displayed in 
Table 3. 


Table 3. Examination of Fit of the Revised Structural Model (n = 319) 



CMIN 

P 

df 

TLI 

CFI 

RMSEA (90% Confidence Interval) 

Revised Structural 
Model 

129.400 

.000 

44 

.962 

.975 

,078(.063~,094) 

Initial Structural 
Model 

126.515 

.000 

42 

.960 

.975 

,080(.064~,096) 

Criteria 




>.90 

>.90 

<.08 


Accordingly, the results of examining effects between self-efficacy in using digital textbooks, subjective 
norms, perceived usefulness, observability, and perceived ease of use are as follows Figure 2. 




Figure 2. Standardized Path Coefficient of Revised Structural Model 

The effect of perceived usefulness on self-efficacy in using digital textbooks was P = .722 (t = 12.214, p 

< .05), that of observability was P = .121 (t = 2.104, p < .05), that of self-efficacy in using digital textbooks 
on perceived usefulness was P = .486 (t = 4.584, p < .05), that of subjective norms was P = .227 (t = 2.880, p 

< .05), that of perceived ease of use was P = .168 (t = 2.264, p < .05), that of perceived ease of use on 
intention of acceptance was P = .254 (t = 3.814, p < .05), and that of perceived usefulness was P = .589 (t = 
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8.483, p < .05). This suggests causal relationships among self-efficacy in using digital textbooks, subjective 
norms, observability, perceived ease of use, perceived usefulness, and intention of acceptance. 

It was also observed that self-efficacy in using digital textbooks and observability affect perceived 
usefulness and perceived ease of use. In addition, self-efficacy in using digital textbooks and subjective 
norms affect perceived usefulness. These factors also affect intention of acceptance. At the same time, since 
perceived ease of use and perceived usefulness simultaneously affect perceived usefulness, we confirmed the 
significance of indirect effects among the variables by using the Sobel test (Kline, 201 1). 

Self-efficacy in using digital textbooks affects perceived ease of use through perceived usefulness (z = 
2.09, p = .03), self-efficacy in using digital textbooks affects intention of acceptance through perceived ease 
of use (z = 3.34, p = .00), self-efficacy in using digital textbooks affects intention of acceptance though 
perceived usefulness (z = 3.77, p = .00), subjective norms affect intention of acceptance through perceived 
usefulness (z = 2.60, p = .00), and perceived usefulness affects intention of acceptance through perceived 
usefulness (z = 2.107, p = .03), at the level of a .05 with indirect effects. However, it was shown that the 
indirect effect of observability on perceived usefulness through perceived ease of use (z = 1.527, p = .12) was 
not significant. 


4. CONCLUSION 

In this study, we investigated causal relationships between elementary school students’ self-efficacy in using 
digital textbooks, subjective norms, observability, perceived ease of use, perceived usefulness, and intention 
of acceptance, based on a modification of Venkatesh and Davis’s Technology Acceptance Model (1996). The 
following implications were derived. 

First, it appeared that the observability and self-efficacy in using digital textbooks affected the perceived 
ease of use but subjective norms had no effect. The effects of self-efficacy of digital textbooks on perceived 
usefulness are consistent with previous research results (Wang & Wang, 2009). This suggests that the higher 
the students’ confidence, the less they experience dissatisfaction with using digital textbooks. In addition, 
observability is consistently shown in previous studies as an effective variable on self-efficacy in using 
digital textbooks, suggesting that the more students observe the desirable examples of teachers using digital 
textbooks, the more the students are able to utilize them as well. 

On the other hand, the result that subjective norms do not affect the perceived ease of use is different 
from our initial hypothesis but is consistent with previous research by Oak (2013). This may be because 
students’ peers and teachers were already using digital textbooks in the existing class and the expectations of 
their peers and teachers cannot affect their utilization of digital textbooks. 

Second, the results of investigating the effects of efficacy in using digital textbooks, subjective norms, 
observability, and perceived ease of use on perceived usefulness, self-efficacy in using digital textbooks, 
subjective norms, and perceived ease of use, affected perceived usefulness but observability did not provide 
significant effects. 

First, the results of self-efficacy in using digital textbooks on perceived usefulness is consistent with 
previous research (Joo et ah, 2013; Albion et ah, 2002). Students who actively utilize new technology with 
positive attitudes recognize digital textbooks as useful. In addition, it appeared that subjective norms 
significantly affect perceived usefulness. The positive perception of friends and teachers using digital 
textbooks positively affected the students’ perceptions of digital textbooks. It was shown in previous studies 
that the perceived ease of use significantly affects perceived usefulness (Davis, 1989; Ong et ah, 2003; 
Rogers, 2003; Venkatesh, 1999). Students perceive digital textbooks as being more useful if they think that it 
is easy to use them. 

On the other hand, the result that observability does not affect the perceived usefulness differs from the 
previous research by Lee, Hsieh, and Hsu (2011) and Seo (2011) but is consistent with that of Chen, Yen, 
and Chen (2009). We can predict that the reasons observability does not affect perceived usefulness is 
influenced by the environment of using a digital textbook. Subjects used digital textbooks only for certain 
courses within a limited period of time (twice a week, and less than two hours) under teachers’ guidelines. 
Additionally, in several schools, students were allowed to use digital textbooks only within a limited space 
and constrained class hours to prevent theft and damage. Therefore, students were not likely to easily observe 
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peers utilizing the digital textbooks after class hours. For that reason, it is predicted that observability was not 
able to affect perceived ease of use in the current study. 

Third, the elementary school students’ perceived usefulness significantly affected the intention of 
accepting digital textbooks. The results are consistent with previous studies (Davis et al., 1989; Kang & Kim, 
2013), suggesting that the more students utilize digital textbooks and believe that a digital textbook is helpful 
to increase learning outcomes, the more easily they are able to accept digital textbooks. Summarizing the 
above research results, we can assume that we need to increase students’ self-efficacy in using digital 
textbooks and observability in order to increase students’ perceived ease of use, self-efficacy in using digital 
textbooks, and subjective norms and perceived usefulness to increase perceived ease of use and perceived 
usefulness. The implications and practical strategies based on the above analysis results are as follows: 

First, it was discovered that self-efficacy in using digital textbooks is a significant variable affecting 
perceived ease of use and perceived usefulness. Therefore, we need to provide enough time for students’ 
orientation and practice regarding how to use digital textbooks to increase their self-efficacy in using them 
prior to introducing them nationwide and the skills and methods of how to use digital textbook should be 
easily provided so that students are able to easily master usage (Kim, 2013). 

Secondly, subjective norms were discovered to have an effect on learner’s perceived usefulness. 
Therefore, it is suggested that efforts be made to increase adoption by announcing and promoting the 
successful cases of using digital textbooks through online boards or communication at home (Joo, Kim, & 
Lim, 2012). 

Thirdly, observability appeared to have positive effects on perceived ease of use. Therefore, teachers 
should provide chances to share their utilization methods or study results through activities or presentations 
to the class. The government should make every effort to provide more opportunities to observe various 
methods and results of utilizing digital textbook in a variety of class promotional activities to increase 
observability (Seo, 201 1). 

Fourthly, the Korean Education and Research Information Service needs to implement efforts to provide 
immediate support and guidelines when problems occur in producing and distributing digital textbooks in 
order to enhance the students' perceived ease of use (Joo et al., 2013). Teacher -centered workshops on how to 
use digital textbooks should be held so that they are able to implement the practice of digital textbook use 
properly for students in the field. 

Finally, we need to create content based on a variety of subjects and activities so that students recognize 
that digital textbooks are helpful in achieving their learning goals; this needs to be carried out to improve the 
digital textbooks’ perceived usefulness. For this, teachers need to provide students with various learning 
activities such as fieldtrips, cooperative learning, and project based learning to utilize digital textbooks in a 
variety of situations and have the chance to share regular uploads of historic animation and human QR codes 
regarding human body structure necessary for learning. In addition, students may perceive digital textbooks 
as useful for learning by having them share learning content and learning results using these textbooks 
through group discussions and presentations (Min, 2013). While a number of previous studies (Gong, Xu, & 
Yu, 2004; Leung & Wei, 2000) have discovered the factors affecting user’s intentions of accepting 
technology based on e-learning or mobile environments, the research focusing on elementary school students’ 
intentions of accepting digital textbooks have not been investigated. Therefore, the current study can be 
differentiated from other research since the main subjects were elementary school students and we 
investigated their self-efficacy in using digital textbooks, subjective norms, observability, perceived ease of 
use, and perceived usefulness and intention of accepting technology, based on Roger’s diffusion of 
innovation theory. 

The current study reflects the educational trends of the nationwide diffusion of digital textbooks. It is 
commendable that the current study tried to find strategies of increasing actual users’ intentions of accepting 
digital textbooks. The limitations and suggestions for further studies are as follows; First, the current study is 
limited in generalizing the research results since the participants were elementary school students selected by 
the Korean Educational and Research Information Service. Therefore, further studies need to broaden the 
targets to various elementary schools, as well as junior and high school environments to increase the 
generalization of the study results. Second, further research needs to consider other influential variables than 
the factors the current study incorporated, such as technical characteristics of system quality and information 
quality, as well as individual characteristics such as individual innovative orientations. 
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ABSTRACT 

The combination of the process of pedagogical planning within the Blended (b-)learning environment with the users’ 
quality of interaction (Qol) with the Learning Management System (LMS) is explored here. The required Qol (both for 
professors and students) is estimated by adopting a fuzzy logic-based modeling approach, namely FuzzyQoI, applied to 
LMS Moodle data from two academic years of two dance disciplines, i.e., the Rare and Contemporary ones. The 
distribution of the estimated Qol across the segmented time-period of the each academic year for each user’s type and for 
each discipline show the beneficial role of Qol to shift the educational scenarios and strategies towards a more dynamic 
design, yet taking into consideration the inherent tendencies and attitudes of the users’ interaction within the b-learning 
context. These are revealed as causal relations in the LMS interaction patterns between users, along with dynamic 
variations in their individual attitude when interacting with LMS. The findings presented here could shed light upon 
designing factors of educational scenarios, in general, but also to those involved in cultural preservation and exploitation 
initiatives, such as the i-Treasures project (http://i-treasures.eu/). 


KEYWORDS 

Blended-learning, LMS, Quality of Interaction, Fuzzy Logic, Rare and Contemporary Dance, i-Treasures. 


1. INTRODUCTION 

The extraordinary development of the Information and Communication Technologies (ICTs) in Education 
has influenced the Higher Education Institutions (HEIs) to adopt online learning solutions/environments. 
Universities and educators are challenged to provide quality in online learning environments (OLE), since 
teaching and learning paradigms are emerging and, in general, students are more open in taking online 
courses. In higher education, there are mainly two well-known modalities for online learning, namely: fully 
online learning and blended(b-) learning courses (Bates and Sangra, 2011). For fully online courses, all 
course materials are provided via the OLE without any face-to-face (F2F) component. For the b-learning 
case, also known as hybrid or combined mode course, both F2F and online class components are considered. 
Globally speaking, many people live their lives in a “blended” way, combining physical and online 
activities/experiences. Here, the b-learning concept not only fits into the contemporary, connected lifestyle, 
but also provide particular gains to students, teachers, and administration, namely: i) increased access and 
convenience; ii) improved learning; and iii) decreased (or more flexible) costs (Stein and Graham, 2014). 

1.1 Interaction in b-learning Environments 

In general, the Learning Management Systems (LMSs) provide educators an environment to place their 
online course materials and for students to receive that education while interacting with other 
students/teachers. However, students’ interactions, attention and communications are seen as relatively low 
in the LMSs (Musbahtiti and Muhammad, 2013). 


103 



ISBN: 978-989-8704-08-5 © 2014 


Anderson’s online pedagogical model (Anderson, 2004) is based on three types of interactions presented 
by Moore (1989), i.e., student-student interaction, teacher-student interaction, and student-content 
interaction. Anderson’s OLE primarily focuses on independent and collaborative learning, highlighting the 
importance of the role of the interaction. It is also known as a model of e-learning, which allows structuring 
and organizing online learning through six particular types of interaction, i.e., teacher-content, content- 
student, student-student, student-teacher, student-content and content-content. Indeed, the two identified 
actors (teacher and student) interact with each other and with the contents. During this interaction, a wide 
variety of activities (synchronous and asynchronous) can be used, based on the Internet (e.g., audio, video, 
conferencing, chats, and virtual worlds). These environments are particularly enriched, promoting the 
development of social skills and collaborative work, as well, at the level of interpersonal relationships 
between the participants (Anderson, 2004). In fact, Anderson’s online pedagogical model can be interpreted 
and analyzed in two separate parts. In the first part, students can interact with the content that is available in 
various formats (especially with Web tools), or they can choose to have their learning in a sequential way, 
i.e., guided and evaluated with the help of the teacher. This interaction can be seen as a community that is 
mainly stimulated by the use of various e-activities, synchronously or asynchronously, supporting social 
interaction, collaborative learning content and the development of interpersonal relationships. In the second 
part, learning tools are based on independent learning; however, even if guided to independent study, the 
students continue to be followed, since they share the same workspace with other colleagues or peers-to- 
peers connections that ensure permanent cooperation and communication interfaces. Considering that both b- 
learning and LMSs opportunities are growing, it is important to understand about the type of interaction 
available in the OLEs and the way it defines the quality of interaction (Qol). According to Parker (1999), the 
lack of interaction is considered an important obstacle to online learning; however, several improvements in 
ICTs over the last years have enhanced the Qol in online environments. From this perspective, there are five 
types of interactions that educators can consider into the curriculum, namely: i) learner to instructor; ii) 
learner to learner; iii) learner to content, learner to tools, and learner to environment (Bastedo and Vargas, 
2014). At the same time, there is a variety of tools that assist in facilitating interaction in online learning. 
These tools have been classified into two different categories, i.e.: asynchronous (e.g., quizzes, discussions) 
and synchronous (e.g., online chat) technologies. Considering the main concepts regarding interactions in 
OLEs and the corresponding tools/resources that facilitate those interactions, it may still be difficult to decide 
when and which tool will fit better into an online course. Here, it is important to select the interactions and 
tools/resources according to the need of the learning goals. In addition, different areas should be considered 
when an online course is created, namely: 1) course content; 2) delivery of instruction; 3) communication and 
interaction; 4) student time spent on learning tasks; and 5) assessment of student learning (Zhu et al., 2003). 
Although the course content may be the same, the delivery of instruction may be given in both modalities 
(synchronous or asynchronous). Among others, the selected assessment activities can be done using chats, 
forum discussion, assignments or quizzes. However, since each OLE is unique, the role of the educator is to 
assess their OLE and determine/adapt if changes are necessary or not. 

Taking into account the users’ interactions (professors’ and students’) through the LMS use within a b- 
learning environment, here, the FuzzyQoI model (Dias and Diniz, 2013; Dias et al., 2014a) is adopted. The 
latter is used as a translator of the knowledge of the experts in the field into fuzzy constructs, and estimates, 
in a quantitative way, a normalized index of the users’ Qol across two consequent academic years at a HEI. 
This process is exemplified here with two dance paradigms, i.e.. Rare and Contemporary Dance disciplines at 
a HEI, serving as a bet-set for the realization of b-learning scenarios within the i-Treasures project (http://i- 
treasures.eu/), related with the capture of the Intangible Cultural Heritage (ICH) and learning the rare know- 
how of living human treasures (FP7-ICT-2011-9-600676-i-Treasures) (Dias et al., 2014b). 


2. METHODOLOGY 

In the effort to develop a successful evaluating system of the user’s interaction with the LMS through the 
Qol, intelligent systems may play an important role, i.e., provide a model of the domain expert’s evaluating 
system, with the promise of advanced features and adaptive functionality (Levy and Weld, 2000). Based on 
the latter, a Mamdani-type (Tsoukalas and Uhrig, 1996) fuzzy logic -based Qol modelling, namely FuzzyQoI 
scheme, was proposed by Dias and Diniz (2013). The FuzzyQoI model constitutes a Fuzzy Inference Systems 
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(FISs) structure that is able to produce evaluative inferences upon input data. In particular, the latter 
correspond to the key-parameters and variables (metrics) of LMS Moodle (https://moodle.org) involved 
within a b-learning environment concerning the user’s interaction with the system, whereas the outputted 
inference forms a quantitative measure of the user’s overall Qol (Dias and Diniz, 2013). 

Generally speaking, the users (professors/students) interact with the LMS and the available 110 LMS 
Moodle metrics are corresponded to 12 categories that serve as inputs to the FIS structure. In an effort to 
efficiently handle the 12 input variables, these are grouped into three groups and a nested sequence of five 
FISs (FIS1-FIS5) is used to form the proposed FuzzyQoI scheme. 

For the construction of the knowledge base of the FuzzyQoI scheme, an expert in the field of analyzing 
LMS Moodle data within the context of b-learning was used, for defining the structure of the membership 
functions used for each FIS and the corresponding IF/THEN fuzzy rules. In particular, a three-level of 
trapezoid membership functions corresponding to Low, Medium and High values, respectively, were used for 
the FIS1-FIS4, whereas a five-level of trapezoid membership functions corresponding to Very Low, Low, 
Medium, High and Very High values were adopted for the final FIS5, increasing, this way, the resolution in 
the segmentation of the universe of discourse in the final FIS5. 

Analytical description of the FuzzyQoI model can be found in Dias and Diniz (2013). 

2.1 Pedagogical Planning 

The organization of educational scenarios during the LMS interaction is facilitated here with the adoption of 
the Pedagogical Planning (PP) (Olimpo et al., 2010). The realization of the latter is achieved by adopting the 
MindMup tool from the i-Treasures Pedagogical Planner (Bottino et al., 2013), which is a scalable cross- 
browser Web-based application developed in PHP, MySQL and lavaScript. The PP is essentially a teacher- 
oriented online tool, yet in the way it is used here, it could serve as a combinatory tool that incorporates both 
designing and planning of the educational interventions and feedback from the realization of the b-learning 
delivered instruction. In this way, causal relations between professors’ and students’ at the level of their 
LMS -based Qol could be identified and professors’ metacognitive processes could be fired towards the 
enhancement of their pedagogical planning and delivery. The PP comprises of both authoring and display 
capabilities, with specially designed functions and interface features in both cases. In particular, target 
population, learning context, content domain, objectives and metrics, along with available tools (such as 
MindMup), are the core characteristics of the PP (http://i-treasures.eu/). 

To exemplify the combination of the FuzzyQoI model with the Pedagogical Planner two cases that 
resemble the use-cases of the i-Treasure project, i.e., the Rare and Contemporary Dances, are used as 
paradigms and described in the subsequent section. 

2.2 The Rare and Contemporary Dances Cases 

Rare Dances at the FHK, actually, belong to the Social Dances discipline, which aims to provide and develop 
ways to dance, able to contribute to a students’ education in a more complete, comprehensive and 
multifaceted way, through the diversity of approaches and multiplicity of perspectives developed in each 
dance form. Moreover, the social dimension and respect of the act of dance are taken into account to enhance 
the knowledge and extend the application domain with multicultural approaches, revealing the nature and 
specificity of their contents. The planning of this discipline aims to construct a place of experience and 
experimentation with different materials, choreographic and contextual, along with specific techniques for 
analysis, leading to “know-how” and the enlargement and consolidation of formal and expressive repertoire 
of the students. 

Contemporary Dances at the FHK are included in the Techniques of Theater Dances discipline, which 
aims to promote the analysis and study of motor vocabulary characteristic of modern and classical dance 
forms. The PP includes practice of standardized modeling steps organized in simple exercises with 
repetitions and chained in sequence dances increasing complexity. Moreover, training skills of observation in 
situations of mutual learning, are also considered, being consistent with the principles and quality of dance 
movements. Figure 1 illustrates the PP of the both paradigms (Rare and Contemporary Dance) in the form of 
the MindMup output, where the principal components, i.e., scientific domains, learning objectives, b-learning 
context, LMS Moodle tools and forms of assessment, are shown in the form of connected branches. 
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Figure 1. The MindMup output of the PP of the Rare and Contemporary Dances with the main elements (i.e., scientific 
domains, learning goals, b-learning context, LMS Moodle tools and forms of assessment). 


3. DATASET USED 

The FuzzyQoI (Dias and Diniz, 2013) was applied to LMS Moodle data from two dance disciplines, i.e., 
Rare and Contemporary Dances, drawn from the Faculty of Human Kinetics (FHK), University of Lisbon, 
Portugal, where the corresponding dance disciplines are realized within the b-learning context. For each 
paradigm (Rare and Contemporary Dance), the 110 LMS Moodle metrics data for two academic years (Yl: 
2009-2010, Y2: 2010-2011) from two teachers (combined teaching) and students from [Rare Dance: 29 (Yl) 
and 57 (Y2); Contemporary Dance: 43 (Yl) and 55 (Y2)], respectively, were used and analyzed. In order to 
identify any possible changes in the users’ interaction behavior correlated with a specific time -period section, 
a time-period segmentation was adopted. The latter has resulted in time-period sections (e.g., semesters (SI: 
2-16, S2: 23-38 weeks), exam periods (1 st : 18-23, 2 nd : 38-46 weeks), interruptions (16-18, 24-25; 30-31 
weeks)) that served as landmarks in 51-week total examined period. 


4. RESULTS AND DISCUSSION 
4.1 The Rare Dance Case 

Figure 2 depicts the estimated Qol and its mean value Qol ± std (grey area) from the FuzzyQoI model for 
the Rare Dance case across the two academic years examined (2009/10-top and 2010/1 1 -bottom). In 
particular, Qol values from the Professors and Students are illustrated in the left and right panels, 
respectively. In Figure 2, the vertical lines denote the specific weeks that define the time-period 
segmentation. 

Focusing at the Professors for the Rare Dance case (Figure 2-left panel), a reduced Qol is seen in both 
ones during the academic year of 2009/10, with Professor 2 being more active with the LMS, compared to 
the Professor 1, exhibiting Qol values around 0.5, both at the first and the second semester. On the other 
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hand, Professor 1 exhibits Qol around 0.4, only at the end of the first semester. This absence of high Qol 
values is also noticeable in corresponding Qol values, which exhibit a clear reduction across the first 
semester and some local increases during the second one. Shifting to the succeeding academic year of 
2010/11, Professor 1 shows increased Qol values (0.4-0. 5) only at the 2 nd exam period (38-46 week), 
whereas Professor 2, like during 2009/10, exhibits increased Qol both at the two semesters, especially at the 
beginning, mid and end of them, accordingly. These local bursts of Qol values are also evident in the 
corresponding Qol values. Since the two professors are the same across the two sequential academic years, it 
seems that, in general, they adopt a conservative approach to the involvement of online material and LMS 
interaction within their curriculum, being its users themselves. Basically, transition points in the time-period 
(such as beginning/ending of semesters, exam period) seem to motivate more the professors to interact with 
the LMS, although the latter is not sustained across the whole academic year. 


RARE DANCE: Estimated Qol 12009/10) ^ DANCE: EslimatMl 0o1 t !t09,10 l 



2 16 18 232425 3081 38 46 

in 1. m i n r 



2 16 16 232425 3031 38 46 
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Figure 2. The estimated Qol and its mean value Qol + stcL (grey area) from the FuzzyQoI model for the Rare Dance case 
across the two academic years examined (2009/10 and 2010/11). Left panel: Professors; Right Panel: Students. The 
vertical lines denote the specific weeks that define the time-period segmentation. 


Looking at the Students for the Rare Dance case (Figure 2-right panel), the absence of interaction across 
the first part of the first semester of 2009/10 (weeks 2-16) is evident (also at the corresponding Qol values). 
Then, a sparse activity of Qol values is seen, maximized at the beginning of the second semester and 
sustained almost across the rest part of the academic year, yet not by all students. This behavioral ‘landscape’ 
is reversed at the next academic year (2010/11), where the LMS activity and the achievement of Qol values 
around 0.5 concentrate at the beginning of the first semester, are reduced as the semester evolves, are 
sustained across the 1 st period of exams and gradually are reduced towards the 2 nd period of exams. It is 
noteworthy, that some students continue to interact with the LMS exhibiting good Qol even after the end of 
the 2 nd exam period (weeks 46-51). By comparing the Qols from the two academic years in the case of Rare 
Dance, and taking into account that the students involved are different yet the two professors are the same, it 
could be seen that in both years, students exhibit higher Qol values at the time-period just after Christmas’ 
interruption (week 18) up to the beginning of the 2 nd period of exams (week 38), following, somehow, a 
hysteresis Qol pattern compared to the professors’ one. 
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4.2 The Contemporary Dance Case 


Moving to the Contemporary Dance case. Figure 3 illustrates the corresponding results as the ones depicted 
in Figure 2, respectively. Nevertheless, in general, the interaction with the LMS is clearly different in the 
Contemporary Dance case, as both Professors are more interactive, compared to the ones from the Rare 
Dance case (Figure 2-left panel). For example, as it is shown in Figure 3-left panel. Professor 2 exhibits 
sustained Qol values around 0.5 across almost the whole period of semesters and exam periods (weeks 4-43) 
for the academic year of 2009/10. Professor 1 follows a similar pattern, yet a more sparse one. It is interesting 
to notice that both of them reduce their Qol with the LMS at the next academic year (2010/11), with 
Professor 1 activating it only in the middle of the 2 nd exam period (week 41) and Professor 2 keep it active 
only until the end of the Easter interruption (week 31). Looking at the Qol results for the case of Students 
(Figure 3-right panel), Qol values around 0.4-0. 5 are sustained almost for the whole duration of the two 
academic years, yet with a more uniform distribution appearing especially across the 2009/10. By comparing 
the Qols from the two academic years in the case of Contemporary Dance, and taking into account that the 
students involved are different yet the two professors are the same, it could be seen that in both years, 
students are almost synchronized with the professors’ Qol pattern, exhibiting increased Qol at the same week 
or the one just after the one noticed in the professors’ case (see for example weeks 3-6, 17-18, 23-25 for the 
academic year of 2010/11). 
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Figure 3. The estimated Qol and its mean value Qol + std (grey area) from the Fuzzy Qol model for the Contemporary 
Dance case across the two academic years examined (2009/10 and 2010/11). Left panel: Professors; Right Panel: 
Students. The vertical lines denote the specific weeks that define the time-period segmentation. 


4.3 Dynamics of Qol 

From the results presented in Figures 2 and 3 it is evident that the FuzzyQoI model provides a useful index, 
i.e., the Qol, to represent the behavior of the stakeholders involved with the online teaching and learning 
processes, that is Professors and Students. Apparently, the distribution of Qol across the whole academic 
year reveals the dynamics underlying within the profiles of both Professors and Students. This is further 
reinforced, when Qol is examined across subsequent academic years (as the two ones examined here) and for 
different disciplines (as the Rare and the Contemporary Dance analyzed here). In this way, ups and downs in 
the Qol of interaction (both from Professors and Students) could be associated with the time-period 
segmentation, revealing the influence of the structure of the academic year, in terms of formal teaching, 
interrupting and examining time periods, to the interaction attitude with LMS. 
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To further elaborate this finding, the distribution of the students’ number that exhibited Qol > 0.1 for the 
two semesters of each academic year and for the two examined cases (Rare and Contemporary Dance) was 
estimated and illustrated in Figure 4, along with the corresponding time-period segmentation. It should be 
noted that this analysis was restricted to the students’ case only, as the number of professors (just two) did 
not allow for any valid statistical perspective. For the Rare Dance case and the academic year of 2009/10 (top 
left panel), the students exhibited zero interaction with the LMS at the first semester, whereas, at the second 
one, they were clearly interacting with the LMS, mostly at the before and during the 2 nd exam period. In 
2010/11, a more balanced LMS interaction is noticed across each semester, exhibiting almost similar patterns 
in both semesters. Interestingly, the latter is also noticed in the case of Contemporary Dance for both 
semesters and for both academic years, in general. 


2009/10 Academic Year RARE DANCE 2010/11 Academic Year 



Semester 2 Semester 2 


Figure 4. The distribution of the students’ number that exhibited Qol > 0.1 for the Rare (top panel) and Contemporary 
(bottom panel) Dance cases, across the two semesters of the two academic years examined (2009/10-left panel and 
2010/1 1-right panel). The vertical lines denote the specific weeks that define the time-period segmentation. 

An interesting finding resulting from Figures 2-4 is the way the discipline affects not only the frequency 
of interaction but also the Qol with LMS, both in Professors and Students’ cases. Clearly, the discipline of 
Rare Dance is restricted to more traditionally defined concepts that follow historical paths and are strongly 
connected to tradition aspects. In that case, LMS-based activities should convey such information as much as 
possible, taking into consideration the nature of each rare dance, as well as the students’ interests aiming to 
provide a balance between intellectual tasks (reading, viewing) and interactive tasks, as well as, between 
group activities and practical experience away from the computer, in order to actively engage students and 
help them to both understand the dance context, as well as, to learn how to dance. Nevertheless, in the case of 
Contemporary Dance, more abstract dance issues are examined, supporting LMS-based interactive tasks, so 
the students could make connections across subject and content areas, thus facilitating their creative thinking 
and enhancing learning in other subjects as well (e.g. music, history, culture, etc.), making the learning 
process creative and more motivating. The findings here support these distinct perspectives between Rare and 
Contemporary Dance disciplines, revealing the way their differences are reflected into the realization of the 
online component of the b-learning context under a common PP (see Figure 1). The latter could be very 
useful for effective designing of educational scenarios within the concept of sustaining the cultural heritage, 
such as teaching rare dances to young generations (New Millennium Learners) and build upon tradition to 
create a contemporary output, using LMS-based b-learning and the Qol metric, as in the case of the i- 
Treasures project (http://i-treasures.eu/). 
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5. CONCLUSION 

The exploration of the dynamics of the Qol with LMS, both by Professors and Students at a HEI b-learning 
context, was attempted here. Two disciplines of dance, i.e., the Rare and Contemporary ones, were taken as 
the analysis focus, and 1 10 metrics from the LMS Moodle users’ interaction across two consequent academic 
years (2009/10 and 2010/11) were drawn and used as input to a recently developed fuzzy logic -based model, 
namely FuzzyQoI. The latter outputs the metric of Qol, translating the expert knowledge to a series of 
fuzzification/defuzzification processes. The distribution of the estimated Qol across the segmented time- 
period of the each academic year for each user’s type and for each discipline revealed causal relations in the 
LMS interaction evolution patterns between Professors and Students, along with dynamic variations in their 
individual attitude within the online learning environment. These findings could be incorporated into 
initiatives that involve the design of educational scenarios (such as i-Treasures), especially for cultural 
preservation, exploration and evolution, such as the b-learning trajectory from rare dances to the 
contemporary ones. 
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ABSTRACT 

Although ubiquitous learning enhances students’ access to learning materials, it is crucial to find out which media 
delivery modes produce the best results for English listening comprehension. The present study examined the effect of 
media delivery mode (sound and text vs. sound) on English listening comprehension and cognitive load. Participants 
were 162 students majoring in Applied Foreign Language at a university in Taiwan. The students were randomly 
assigned to either single mode (sound) or double mode (sound and text). The research questions are (a) whether students 
learning with double mode outperformed students learning with single mode in listening comprehension; and (b) whether 
students learning with double mode hold less cognitive load than students learning with single mode. If the answers to 
these questions are affirmative, then the modality effect occurs and the redundancy effect does not occur. The results 
demonstrated that (a) text significantly enhanced English listening comprehension and lowered cognitive load; (b) 
students with higher English listening comprehension held lower cognitive load, and vice versa; (c) text was added no 
benefit to schema construction in long-term memory; and (d) complex media deliveries were not necessarily helpful to 
learning. Results (a) and (b) confirmed that the modality effect occurred, and the redundancy effect did not occur in the 
present study. 


KEYWORDS 

U-Learning, Media Delivery Mode, English Listening, Cognitive Load 


1. INTRODUCTION 
1.1 Background 

Listening comprehension is difficult for foreign language learners because it is a continuous process that 
requires learners to understand messages while listening to them and sometimes can lead to a heavy cognitive 
load. In order to get rid of listening barriers, various media delivery modes and ubiquitous learning activities 
should be implemented in the instruction because they are convenient and can enhance students’ learning 
motivation and learning performance (Liu & Chu, 2010). Ubiquitous learning involves a context-aware 
environment that users engages in with some mobile devices (Tan, Liu, & Chang, 2007). Hence, learners 
learning in an authentic situation will have better English listening comprehension due to the enhanced 
sensory stimulation. 

The impacts of different media delivery modes on learning remain inconclusive. According to the 
working memory model proposed by Baddeley (2000), working memory can simultaneously receive 
information from different channels, such as auditory and visual. Thus, when the efficiency of the working 
memory is improved, learning performance will also be enhanced. However, Kalyuga, Chandler and Sweller 
(2000) suggested that some multimedia learning software can lead to cognitive load, which affects learning 
performance negatively. Sweller (2005) also argued that unnecessary or repeated multimedia messages can 
result in a redundancy effect, which negatively affects learning performance. 
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Multimedia helps learners learn, but different media delivery modes affect learners’ cognitive load 
differently (Mayer & Moreno, 2003; Plass, Chun, Mayer, & Leutner, 2003). Sweller (2007) mentioned that 
the purpose of instructional design is to assist learners to store information in long-term memory. This 
implies that the way teaching materials presented to learners is a key to instructional design. Attention to 
cognitive load is a critical concern for instructional designers when designing multimedia teaching materials 
because unnecessary multimedia messages will worsen learning performance by increasing working memory 
load and interrupting information processing (Sweller, 2007). 

Multimedia instructional systems have been widely applied in teaching and learning, but the media 
delivery mode that is best for English listening comprehension remains uncertain, and whether unnecessary 
information led to cognitive load for learners also remains inconclusive. According to the studies done by 
Jones and Plass (2002) and Diao, Chandler and Sweller (2007), learners learning with double mode (sound 
and text) outperformed learners learning with single mode (sound) and had lower cognitive load. Studies 
related to foreign language learning and cognitive load are mostly about digital learning environments. 
Hence, the present study examined the effect of media delivery mode on listening comprehension in a 
ubiquitous learning environment to see if there were any differences from the studies on digital learning 
environments. Which media delivery mode can efficiently help learners store information in long-term 
memory is another issue to be further examined. 

1.2 Research Purpose and Questions 

The present study aimed to examine the effect of media delivery mode (single mode: sound; double mode: 
sound and text) on English listening comprehension and cognitive load in a ubiquitous learning environment. 
The research questions include: 1. Are there any significant differences in English listening comprehension 
between two media delivery modes? 2. Are there any significant differences in cognitive load between two 
media delivery modes? 3. Are there any significant correlations between English listening comprehension and 
cognitive load? 


2. METHOD 

2.1 Participants 

Participants were 162 university students in Taiwan, aged from 18 to 23, majoring in Applied Foreign 
Language with the same instructor. These participants were randomly assigned to either single mode group or 
double mode group. There were 82 students in the single mode group and 80 students in the double mode 
group. The participants had similar academic proficiency because all students had been assigned to the 
university based on their scores obtained from the Joined College Entrance Examination. Before the 
experiment started, all the participants had been familiarized with PDA. 

The ubiquitous learning activity in the present study was held at the Taipei Zoo. The Taipei Zoo is one of 
the main Natural Science Education centers in Taiwan, where people can acquire knowledge about animals 
and nature. Since the topic of the learning activity was related to animals, it was more appropriate for learners 
to learn in the zoo. Learning in the zoo enabled learners to experience an experiential learning. With the 
support of PDA, teaching efficiency and learning motivation are enhanced, which can be a contribution to 
education. The possible career opportunities for students majoring in Applied Foreign Language are tour 
guides, who can speak more than one second language, and foreign language teachers. So, the experiment in 
the present study provided an opportunity for the participants to visit the future workplace. 

2.2 Research Design 

The independent variable in the present study was media delivery mode, which were single mode (sound 
only) and double mode (sound and text). The dependent variables were learners’ English listening 
comprehension and cognitive load. There were two tests in the present study. The first test was administrated 
to the participants immediately after the ubiquitous learning activity was over for examining their listening 
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comprehension. The covariate variable was English listening proficiency which was examined by General 
English Proficiency Test (GEPT). 

As shown in Table 1, participants in both groups took GEPT as the pretest. In the ubiquitous learning 
activity, participants in the single mode group learned with sound-only materials and the double mode group 
learned with sound-and-text materials. After the learning activity, both groups took English listening tests 
and cognitive load questionnaire as the posttest. 


Table 1 . Research design (ubiquitous learning) 


Group 

N 

Pretest 

Experiment 

Posttest 

Single 

82 

General English 

_ Prnfipipnpv TpqI 

Sound-only 

English listening material 

English listening test 

Double 

80 

r luiiLiciic y i cm 

(GEPT) 

Concurrently sound and text 
English listening material 

Cognitive load 


2.3 Research Design 

There were four stages in the experiment including pretest (first week), training (second week), and 
intervention and posttest (third week), as shown in Table 2. 

Table 2. Research design 

Stage 

Description 

Pretest 
(First week) 

The pretest was administrated during the class (1.5 hours): 

1. The instruction for the test was given by the instructor (0.5 hour). 

2. Students took listening test from GEPT (1 hour). 

Training 
(Second week) 

The training was provided during the class (3 hours): 

1. Introduction of ubiquitous learning, PDA (HP IPAQ 112 Classic, 3.5 inch screen) 
and GPS, including practical experience on using PDA and GPS. 

2. Instructor provided key words that would be included in the material. 

Intervention and 
Posttest 
(Third week) 

Ubiquitous learning and test in the zoo (4 hours): 

1. Students were randomly assigned into two groups with either single or double mode 
material. 

2. Students had their PDA to be connected with GPS. 

3. The system asked students to enter their student ID. 

4. Exploration of animals would be displayed on the screen. Students could see the 
map of Africa area in the zoo and their current location. Each animal was marked 
on the map and students could decide the listening order by their preferences. The 
system guided students to the target by GPS. 

5. When students arrived in the observed area, the system would display its material 
automatically by GPS and ask students if they wanted to start the listening or not. 

6. Students were presented to English listening material by clicking on the button 
"Play". 

7. After the speech sound played, students needed to click on the button "Next Page" 
for the test page. 

8. After taking the test, students would then continue to the next animal with the steps 
mentioned above. 

9. Students were required to fill-in the cognitive load rating scale. 
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2.4 Instrument 

2.4.1 English Listening Proficiency Test 

The General English Proficiency Test (GEPT) was utilized in the present study to determine students’ 
proficiency in English listening since the GEPT was a graduation requirement for the students majoring in 
Applied Foreign Language. There were 20 multiple-choice questions given by sound speech in the test. Each 
question was worth five points, and the total possible score for the test was 100. 

2.4.2 English Listening Material and Test 

A total of four animals, including elephant, lion, monkey and giraffe, were chosen from the Africa area in the 
Taipei Zoo. Each animal was described by a passage, so there were a total of four passages in the test. The 
English listening training system was developed by the research team in the present study. The content of the 
teaching material and the test were adapted from the website of National Geographic 
(http://animals.nationalgeographic.com/) and San Diego Zoo 

(http://www.sandiegozoo.org/animalbytes/index.html), as shown in Figure 1 and 2. The spoken time for each 
passage, with length from 180 to 220 words, was about three minutes. There were five multiple -choice 
questions for each passage, which required students to answer (without time restriction) after listening to 
each passage. There were 20 questions in the test and one point for each question. 



Figure 1. PDA screen of single mode Figure 2. PDA screen of double mode 

2.4.3 Cognitive Load Rating Scale 

The cognitive load rating scale by Yeung, Lee, Pena, and Ryde (2000) was adopted in the present study, as 
shown in Table 4. The rating scale included four aspects, which were difficulty, incompetence, negative 
affect and lack of effort. The reliability for each aspect was ranged from 0.78 to 0.93, which was acceptable. 
The factor loadings for all the items were greater than 0.5 and the four factors accounted for more than 50% 
of total explained variance. Therefore, the validity of the rating scale was satisfied. The participants were 
required to rate themselves on a 5 -point Likert-type scale with response options from 1 (extremely agree) to 5 
(extremely disagree). The reliability coefficient of the measures of cognitive load was .814, as measured by 
Cronbach’s a, which was acceptable. 
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3. RESULT AND DISCUSSION 

3.1 The Effect of Media Delivery Mode on English Listening Comprehension 
and Cognitive Load (Research Question 1 and 2) 

As shown in Table 3, for English listening comprehension, learners in the double mode group outperformed 
learners in the single mode group. For cognitive load, learners receiving single mode possessed higher level 
than learners receiving double mode. As shown in Table 4, Wilk’s A showed a significant result, indicating 
that learners in both groups had significant differences in at least one dependent variable (English listening 
comprehension or cognitive load). The analysis of two-way MANCOVA, with covariance of English 
listening proficiency, showed that there was a significant difference in listening comprehension ( p < .05) 
between the two groups, indicating that learners in the double mode group outperformed learners in the 
single mode group. There was a significant difference in cognitive load (p < .05) between the two groups, 
revealing that the single mode group had higher cognitive load than the double mode group. Both groups had 
significant differences in listening comprehension and cognitive load, but the estimated effect size for 
listening comprehension (if = 0.117) was greater than cognitive load (if = 0.033). This implied that media 
delivery mode had more impacts on listening comprehension than on cognitive load, and both groups had 
more differences in listening comprehension than in cognitive load. 


Table 3. Descriptive statistics for listening comprehension and cognitive load 


Aspect 


Single mode 



Double mode 



M 

SD 


M 

SD 


English listening 
comprehension 

6.667 

1.916 


8.099 

2.262 


Cognitive load 

51.617 

20.458 


44.621 

19.907 


Table 4. Two-way MANCOVA summary on listening comprehension and cognitive load 

Wilk’s A 
(Sig.) 

Source 

Aspect 

Type III 
Sum of Squares 

df 

Mean 

Square 

F Sig. 

Eta. 

Squared 


Covariance 

Listening 

comprehension 

66.780 

1 

66.780 

16.878* 0.000 

0.112 



Cognitive load 

6183.521 

1 

6183.521 

16.993* 0.000 

0.113 


Between- 

Listening 

comprehension 

70.070 

1 

70.070 

17.709* 0.000 

0.117 

0.877 

group 

Cognitive load 

1671.568 

1 

1671.568 

4.594* 0.034 

0.033 

(0.000) 

Within- 

group 

Listening 

comprehension 

530.197 

134 

3.957 




Cognitive load 

48760.674 

134 

363.886 





Listening 

8187.000 

137 





Total 

comprehension 






Cognitive load 

372151.143 

137 





*p < 0.05 


3.2 The Correlation between English Listening Comprehension and Cognitive 
Load (Research Question 3) 

A Pearson Correlation was performed in the present study to examine if there was a correlation between 
English listening comprehension and cognitive load. English listening comprehension and cognitive load had 
a significant negative correlation (r = -0.393 ’ p < 0.001). 

In ubiquitous learning environment, learners who performed well in English listening comprehension held 
lower cognitive load, and vice versa. This result supported the cognitive load theory and confirmed most 
study results on cognitive load (Chung, 2008; Diao et al., 2007 I Diao & Sweller, 2007 I Jones & Plass, 
2002) that learners had low cognitive load performed well. 
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3.3 Discussions 

In the ubiquitous learning environment, learners in the single mode group held higher extraneous cognitive 
load due to the lack of text support. On the other hand, learners in the double mode group possessed lower 
extraneous cognitive load because they got the support from text. To some learners learning with double 
mode, they only needed to overcome intrinsic cognitive load from the material itself. Hence, learners learning 
with double mode outperformed learners learning with single mode. For the double mode, the modality effect 
occurred, but the redundancy effect did not. These results confirmed some study results on cognitive load 
(Diao, et al., 2007; Jones & Plass, 2002). A study by Chung (2008) revealed that learners with high English 
proficiency held lower cognitive load when they learned with double mode. So, the result in the present study 
that learners who learned with double mode had lower cognitive load could be explained by their sufficient 
English proficiency. Based on the result of the present study, text enhanced students’ listening 
comprehension and lowered their cognitive load. At the same time, the result also confirmed the viewpoint 
by Baddeley (2000) that information could be received from both visual and auditory channels for increasing 
the capacity of the working memory and helping students learn, which referred to the modality effect from 
the cognitive load theory (Sweller, 2005). 


4. CONCLUSION AND IMPLICATION 

Although text significantly enhances English listening comprehension, learning without reviews is difficult 
for learners to build up schema in long-term memory. So, instructors who taught English listening are 
suggested to provide teaching materials with double mode (simultaneous sound and text) for facilitating 
learners’ listening comprehension. After the listening class is over, instructors should provide review 
materials with single mode (sound only) for assisting students to build up schema. 

English listening comprehension and cognitive load held a significant negative correlation. Based on 
some relevant studies (Pawley, Ayres, Cooper, & Sweller, 2005; Paas et al., 2003), extraneous cognitive load 
can be lowered by appropriate instructional designs and learning activities. So, instructors should pay much 
attention to the impact of cognitive load because learners without unnecessary information in the working 
memory learn efficiently. 

The comparisons among single mode, double mode and triple mode, such as sound, text and image, with 
different learning environments, including traditional learning and multimedia digital learning, can be 
included in the future studies. Also, the presentation modes of text can be further categorized into 
synchronization and non-synchronization to examine the effect of text presentation mode on learning 
performance and cognitive load. For cognitive load, extraneous cognitive load can be lowered easily by 
instructional design, so it is appropriate to be a dependent variable in the experiment. Therefore, the present 
study focused only on examining the effect of two different presentation modes on extraneous cognitive load. 
Intrinsic cognitive load and germane cognitive load can be considered as the other dependent variables in the 
future studies. 

Learners’ prior knowledge or proficiency would be a main factor for determining an appropriate media 
delivery mode for learners (Chung, 2008). English proficiency was a covariance in the present study. Prior 
knowledge or learners’ characteristics, such as learning styles and media preference, can be other 
independent variables in the future study. Furthermore, the interaction between learners’ prior knowledge and 
media delivery mode and its effects on learning performance and cognitive load with two-way ANOVA were 
suggested for the future studies. 

The ubiquitous learning activity in the present study was held outdoors, so some students’ listening 
comprehension would be influenced negatively by the crowd, weather or any other exterior factors. 
Researchers are suggested to hold outdoor learning activities on campus for the future research. If researchers 
hold the learning activity in a public place, then hold on non-holidays for preventing unexpected 
interruptions. 
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ABSTRACT 

Numerous e-learning 2.0 studies have advocated the use of social networking sites for educational purposes, but only a 
few of them have observed social networking sites as an instrument for specific learner skill development. This paper 
discusses a study which addresses the motivation and challenges associated with the introduction of the social networking 
tool Elgg for the peer-review activity related to students’ projects evaluation. In our study, Elgg was utilized to support 
the development of skills related to (a) peer-reviewing, (b) collaborative work on research projects and their online 
presentation, and (c) critical analysis of research proposals and reports. This report therefore aims to contribute to those 
areas of learner skills’ development as well as to the evaluation of Elgg as an e-learning 2.0 tool. 


KEYWORDS 

Social networking site, Elgg, peer review, e-learning 


1. INTRODUCTION 

During their average day students spend a lot of time using different social media tools (e.g. Facebook, 
Youtube, Twitter, Instagram, Google tools etc.). Similar to (mobile)phones, listening to music CD’s or 
watching television as their predecessors of two decades ago, such tools have become a part of students’ 
daily social interaction and media consumption related activities. Due to the proliferation of smartphones and 
affordable internet connections, today’s students tend to spend a lot of their time online and, with the use of 
various social media, most of them remain connected with their friends (via Facebook, Skype, Viber, 
WhatsApp etc.) during their active hours. 

Social networks provide a means for posting (announcing, presenting, delivering) and retrieving (finding, 
following, encountering, discovering) different kinds of information. Owing to them, it is possible to track 
what our social connections are doing, what the latest things that everybody is talking about are, what our 
connections like or recommend, what they axe following, or what they think is worth sharing with others. In 
fact, these days students depend closely on the use of online social tools as an important means for everyday 
interaction with their peers. 

The prevalence of social networks use presents challenges for contemporary instructors who wish to 
appropriately exploit the potential of these social tools in a concrete learning setting. It must be noted that 
recent developments in e-learning have reinvigorated the higher education environment, opening up new 
opportunities for learning. However, in e-learning used in a conventional way as part of a hybrid course 
many of its potentials are often not fully exploited and are sometimes even regarded as a minor disturbance. 
For instance, having to attend face-to-face lectures in a traditional classroom, while being able to 
simultaneously or subsequently download representative presentations and PDF files of those lectures from 
the learning management systems (LMS), may result in students’ reduced motivation for listening to the 
lecturer and also for their active participation in class. It must be emphasized that when traditional classroom 
lectures are supplemented with the use of LMS merely as content repository, its prospective use as a place 
for meeting with peers and collaboration among students is generally omitted. On the other hand, it has been 
suggested that in the hybrid teaching approach the concepts of community, collaboration and interaction 
among users should be introduced whenever it is appropriate to overcome limitations of conventional use of a 
virtual learning environment or LMS (Zhou, Sabino & Rodrigues, 201 1). 
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As educational technology researchers, in this paper we aimed to examine how social networking tools 
could be effectively integrated with other university course resources, as well as included in the pedagogical 
design in which the use of those tools is one of course completion requirements in an effort to enhance 
students’ online learning and collaboration. 

In our research and hybrid teaching approach the social networking site Facebook was not included so as 
to reduce the risk of unnecessary distractions, like receiving messages or contact requests from friends who 
do not belong to the class. Such widely used social networking sites can be disadvantageous for education 
purposes owing to news feeds which are unrelated to course content, commercial advertisements and easy 
access to online games, among others. Therefore our decision in the academic year 2012/2013 was to 
implement the social networking tool Elgg in the course entitled Computer-Mediated Communication at the 
Faculty of Organization and Informatics, University of Zagreb. In this course Elgg was used for the 
organization of students’ projects, team collaboration and project related collaborative writing, as well as for 
publication of project proposals and gradually improved reports in combination with peer-to-peer review 
activity at each of the project development steps. The aim of this paper is to present the results of the evaluation 
of learner experiences in the project related peer-review activity using the social networking site Elgg. 


2. REVIEW OF RELEVANT LITERATURE 
2.1 Social Networks in Learning 

Social networks provide a possibility to interact, communicate and collaborate online in innovative, 
unconventional and complex patterns. Boyd and Ellison (2007) describe social networking sites (SNSs) as 
technologies that enable public articulation of social connections whether they are pre-existing networks or 
newly created ones based on some common interest. Numerous authors report that the main reason for using 
social networking sites is social presence (Cheung, Chiu and Lee, 2011), as they allow instant 
communication and connection with our friends, family members or colleagues. Social presence is also 
reported to be an essential component of education and learning (Brady, Holcomb and Smith, 2010). The use 
of SNSs can provide a lot of benefits for learners since they “encourage peer-to-peer dialogue, promote the 
sharing of resources, facilitate collaboration, and develop communication skills” (Siemens and Weller, 201 1). 
Finally, some researchers have concluded that in the educational context social networking is seen not only as a 
possibility, but also as a necessity (Silius, Miilumaki, Huhtamaki, Tebest, Merilainen and Pohjolainen, 2010). 

Several authors have reported the benefits of SNSs in distance education courses as a technological tool 
for improvement of online communications among students (e.g. Brady, Holcomb and Smith, 2010). The 
positive use of social networks in team projects (Nygard, Bender, Walia, Kong, Gagneja and Lenoue, n.d.) 
and language learning (Blattner and Lomicka, 2012) have also been reported. Although Facebook is one of 
the most popular social networking sites and students have a lot of experience using it, several research 
papers have reported that students did not feel safe and comfortable using Facebook in the educational 
context as their privacy might be revealed (Wang, Woo, Quek, Yang and Liu, 2012). Other authors argue that 
although Facebook has a potential to promote collaborative and cooperative learning, further research 
regarding how it can affect learning outcomes is necessary (Irwin, Ball, Desbrow and Leveritt, 2012). 

Besides Facebook, there are other general-purpose SNSs containing all the necessary social features that 
could be applied in different contexts. Such sites provide a unique opportunity for educators to enhance the 
sense of belonging to a certain community and encourage interactions among students, which also lead to 
creation of new knowledge (Brady, Holcomb and Smith, 2010). In the literature there is a strong emphasis on 
the need for technological innovations to be accompanied by pedagogical progress in order to be successful 
within an educational setting. Accordingly, we decided to use the Elgg tool for a specific pedagogical 
activity, i.e. peer-reviewing. 
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2.2 Peer-review Activity 

Peer review activity basically refers to an evaluation of someone ’s work by one or more peers of similar 
competences. The purpose of this activity is to provide the learner with an opportunity to improve their 
reading skills, writing skills, critical thinking and to learn how to effectively collaborate with other students 
by giving and receiving feedback. The ability to produce quality feedback is considered to be one of the 
fundamental skills of graduate students that should receive considerable attention in higher education 
curriculum (Nicol, Thomson and Breslin, 2014). Peer-review activity results in gains both for the writer and 
receiver of feedback (Cho, Schunn and Kwon, 2007). Lundstrom and Baker (2009) revealed that the benefit 
for feedback writers was more significant than that of its receivers. There are several tools which can be used 
for peer-reviewing and some of them were found to bring positive outcomes (Sondergaard, 2009; Chen, 
2012). Indeed, according to Sondegard (2009), “if used properly, peer reviewing can become an important 
step towards the creation of a culture of mutual support amongst the students in a class”. The goal of our 
research was to test the application of the social networking tool Elgg for peer-review activity and investigate 
students’ opinions and experience regarding its use. 


3. RESEARCH AND METHOD 

3.1 Research Questions 

The goal of our research was to identify, describe, and understand students’ experiences in peer-reviewing of 
student’s projects which was facilitated through a social networking platform. In particular, our research 
questions were: 

• What were the students’ experiences performing an online peer-review activity using the social networking 
platform Elgg? 

• How did students perceive the use of the social networking platform Elgg for project peer-review activity? 

3.2 Study Context 

This study was conducted in the winter semester of academic year 2012/2013 with students of a graduate - 
level hybrid course entitled Computer-Mediated Communication at the Faculty of Organization and 
Informatics , University of Zagreb. The Elgg social networking tool was installed on a college server for e- 
learning projects and administered by the first author of this paper. After registration to Elgg, students had to 
personalize their profile page (short introduction, their interests and likes). Most of the students had known 
each other from earlier years of study and many of them had also attended other courses together. 

The authors of this paper had already had considerable experience of using different Web 2.0 tools in 
higher education, as well of using SNSs such as Ning (Bubas, Coric and Orehovacki, 2010). They had also 
been using Mahara ePortfolio, Wordpress blog and MediaWiki engine in educational settings. The challenge 
in this research was to experiment with the use of Elgg beyond its evaluation as a tool. In other words, we 
also intended to examine the peer-reviewing online activity in practice with the use of Elgg, its 
appropriateness and effectiveness in the development of critical thinking, as well as to obtain students’ 
feedback on their experience using the tool. 

In the academic year 2012/2013, the Computer-Mediated Communication course was enrolled by 65 
students. It was an elective course in the first year of the graduate study. As an online activity for this course 
the students needed to form groups in Elgg that were named according to their project title. The students had 
previously selected their project topics by themselves and discussed them with the teachers regarding topic 
appropriateness and possible improvement. The project topics were chosen by the students (with some 
teacher guidance) on the basis of recent scientific literature (e.g. the journal Cyberpsychology, Behavior and 
Social Networks ) and were related to broad thematic areas of the hybrid course, including internet addiction, 
online gaming, online relationships, social network use by different age groups etc. After the students and 
lecturers agreed on the project topic, each team of students was required to develop 5 blog posts which 
represented 5 consecutive versions of their project proposal and results presentation. Each version was 
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available for peer-reviewing upon its completion on a fixed date. Peer reviewing activity was obligatory and 
students needed to choose two other projects which they would follow (i.e., read and reflect upon) until the 
end of the semester. A timetable was set containing dates on which students would need to present each 
version online and in class, as well as provide their reflections on other’s projects. After the project was 
presented and new project presentation versions were available, students’ peers were required to post their 
reflection with some help and guidelines from the teacher. In that way the successive project versions were 
developed and progressed fairly well. In the end, only the final - fifth - version of the project was submitted 
for assessment/grading by the teacher, without the earlier project versions. The main motto for promoting 
peer feedback was “Help your classmates to develop better project versions!” 

3.3 Data and Results 

After the semester and grading had finished we offered the students a possibility to reflect on the hybrid 
course design, on the project peer-review activity, and also on Elgg as a tool to perform collaboration, project 
presentation and peer-review. The participation in the survey was not obligatory since at that time students 
also had to complete several surveys in other courses related to teachers’ performance, so we expected that 
they may not be enthusiastic about the obligation to fill in another survey. Nevertheless, 33 students provided 
their feedback (19 female and 14 male students). It must be noted that out of 33 students in our survey, only 7 of 
them had participated in peer-review activity before. For others this was their first-time peer evaluation experience. 

As mentioned above, after the course completion we applied an online survey with 16 questions that were 
accompanied with a five point Likert -type scale for measurement of students’ responses regarding their 
experience of participating in the peer-review activity. Students were able to provide their answers to each 
survey item on a scale ranging from 1 - Strongly Disagree to 5 - Strongly Agree. Most of the items and 
frequencies of students’ answers are shown in tables 1-3. On the whole, the majority of students from our 
sample found the feedback they received or provided useful and delivered with good intentions, stating that, 
on average, they were satisfied with the peer-review procedure and experience. This was a confirmation of 
the pedagogical value of the peer-review activity and its possibility to be effectively used in the educational 
context. More detailed analysis of survey results regarding particular groups of survey items is given below. 


Table 1. Responses to survey questionnaire items regarding the acceptance of others’ comments in peer-review activity 


Statement 

Answer 

Number 

Percentage * 

1. 1 find comments that our project received to be useful. 

1 - Strongly Disagree 

1 

3% 


2 

5 

15% 


3 

2 

6% 


4 

13 

39% 


5 - Strongly Agree 

12 

36% 

2. We changed some parts of the project because of the 

1 - Strongly Disagree 

2 

6% 

comments that we received. 

2 

2 

6% 


3 

8 

24% 


4 

3 

9% 


5 - Strongly Agree 

18 

55% 

3. Because of reviewers who followed our work I was 

1 - Strongly Disagree 

6 

18% 

motivated to make each project version better than the 

2 

3 

9% 

previous one. 

3 

9 

27% 


4 

8 

24% 


5 - Strongly Agree 

7 

21% 

4. The intention of our project reviewers was to give us 

1 - Strongly Disagree 

3 

9% 

comments/reviews to help us to reveal possible gaps 

2 

0 

0% 

early, so that in the end we could have a quality, well- 

3 

4 

12% 

designed project. 

4 

14 

42% 


5 - Strongly Agree 

12 

36% 


* Note: The total for each statement in the column "Percentage ’’ may differ from 100% because of rounding to the nearest integer. 
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The data presented in Table 1 indicates that the majority of students consider the comments that they 
received by peer review to be useful (75% of responses agree or strongly agree with regards to the first 
survey item in Table 1). Also, most of the students (64%; item no. 2) stated that they made changes to their 
project according to the comments that they received, and almost half of them (45%; item no. 3) stated that 
because their successive project versions were followed by the reviewers they were motivated to repeatedly 
improve them. Finally, as many as 78% considered the intention of project reviewers as positive and directed 
toward overcoming the shortcomings and possible gaps (see responses to item no. 3 in Table 1). It must be 
noted that if the survey items 1-4 in Table 1 were considered as a separate self-assessment scale the internal 
consistency (Cronbach alpha) of this scale measured on our subjects (N=33) would be 0.83, which indicates 
good reliability. 


Table 2. Responses to survey questions regarding skills development of participants in peer-review process 


Statement 

Answer 

Number 

Percentage * 

1. I read reviews that other students wrote for other 

1 - Strongly Disagree 

4 

12% 

projects in order to find ideas and inspiration for my 

2 

6 

18% 

own reviews. 

3 

6 

18% 


4 

11 

33% 


5 - Strongly Agree 

6 

18% 

2. Reviews of other students that were related to projects 

1 - Strongly Disagree 

8 

24% 

other than mine helped me to improve/change the 

2 

7 

21% 

project of our team. 

3 


30% 


4 

3 

9% 


5 - Strongly Agree 

5 

15% 

3. The experience of participating in peer reviewing 

1 - Strongly Disagree 

5 

15% 

helped me in the development of critical thinking in the 

2 

4 

12% 

analysis of textual content. 

3 

6 

18% 


4 

13 

39% 


5 - Strongly Agree 

5 

15% 

4. The experience of participating in peer reviewing 

1 - Strongly Disagree 

6 

18% 

helped me in the development of my scientific paper 

2 

2 

6% 

writing skills. 

3 

9 

27% 


4 

10 

30% 


5 - Strongly Agree 

6 

18% 


* Note: The total for each statement in the column “ Percentage ” may differ from 100% because of rounding to the nearest integer. 


As can be seen from the data in Table 2, the collaborative learning (side) effects of participation in the 
peer-review process may have positively influenced the motivation and mental schema (“idea”) development 
for the peer-reviewing process for a considerable proportion of participants in our survey (51%; see item 1 in 
Table 2). About the same positive effects were reported by respondents regarding the critical thinking ability 
in the analysis of the texts of research proposals and project reports of peers (54% of the respondents agreed 
with the content of item 3 in Table 2). Finally, the experience of participation in the peer-review process on 
the whole was considered helpful by almost one half of the respondents (48% of them, to be exact; see item 4 
in Table 2) in developing their skill of writing a scientific paper. However, only a small proportion of 
students (24%; item 3) stated that reading the reviews of other projects helped them to improve their own 
project. As for the items presented in Table 2, if the survey items 1-4 in this table were considered as a 
separate self-assessment scale related to skills development in peer-review activity, the internal consistency 
(Cronbach alpha) of this scale would be 0.86 (N=33), which also indicates its potentially good reliability. 

The final section of the evaluation survey regarding the use of peer review contained items related to 
several aspects of the use of Elgg (see Table 3.). The data obtained from the survey suggest that students 
were predominantly satisfied (63% of subjects; item 1 in Table 3) with the use of the social networking tool 
Elgg during their project creation, as well as with its use for the peer-review activity (69% of subjects; item 4 
in Table 3). The Elgg’s blog tool also received positive evaluation by the majority of students for both the 
project activity (57% of subjects; item 2 in Table 3) and peer-review (72% of subjects; item 3 in Table 3). 
Finally, as in the case of items presented in Table 1 and Table 2, if the survey items 1-4 in Table 3 were 
considered as a self-assessment scale related to evaluation of the social networking tool Elgg for peer-review 
activity, the internal consistency (Cronbach alpha) of this scale would be 0.90 (N=33), which indicates very 
good reliability. 
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Table 3. Responses to survey questions regarding the evaluation of Elgg for peer-review activity 


Statement 

Answer 

Number 


1. I am satisfied with the use of social networking tool 

1 - Strongly Disagree 

2 

6% 

Elgg during project creation within the CMC course. 

2 

3 

9% 


3 

7 

21% 


4 

12 

36% 


5 - Strongly Agree 

9 

27% 

2. I consider that the use of a group blog within the 

1 - Strongly Disagree 

3 

9% 

social networking tool Elgg is suitable for the project 

2 

5 

15% 

activity. 

3 

6 

18% 


4 

12 

36% 


5 - Strongly Agree 

7 

21% 

3. I consider that the use of a group blog within the 

1 - Strongly Disagree 

1 

3% 

social networking tool Elgg is suitable for the 

2 

3 

9% 

performance of peer-review activity. 

3 

5 

15% 


4 

16 

48% 


5 - Strongly Agree 

8 

24% 

4. I am satisfied with my experience of using the social 

1 - Strongly Disagree 

2 

6% 

networking tool Elgg for the performance of peer-review 

2 

2 

6% 

activity. 

3 

6 

18% 


4 

16 

48% 


5 - Strongly Agree 

7 

21% 


* Note: The total for each statement in the column "Percentage ’’ may differ from 100% because of rounding to the nearest integer. 


As the final step in the analysis of our survey data we performed factor analysis of the responses to the 
items presented in tables 1-3. The results of the principal components confirmatory factor analysis with 
varimax rotation generally confirmed the categorization of items into three groups, i.e. “Acceptance of 
others’ comments in peer-review activity” as factor FI, “Skills development of participants in the peer- 
review process” as factor F2 and “Evaluation of Elgg for peer-review activity” as factor F3. The result of this 
factor analysis, together with the value of Cronbach alpha coefficients that were calculated as if the items in 
tables 1-3 were considered as separate self-assessment scales (the values were 0.83, 0.86 and 0.90, 
respectively), indicate that these items could be useful for similar research regarding evaluation of peer- 
review activity that is supported by the use of social networking tools. 

Although Elgg provides a lot of special features (like wall posting, bookmarking, posting video or 
photographs etc.) with the possibility of unique sharing of information modes, the use of these features was 
fairly modest during the peer-review activity in our study (as the students mainly used chat and blog). In fact, 
the students were predominantly concentrated on the fulfillment of their assignments and on the peer-to-peer 
communication which was necessary for the ongoing project related activity. This can be explained by the 
fact that in the hybrid course the students were able to meet each other regularly in class, so they did not need 
additional types of interaction. 


4. FINDINGS AND CONCLUSION 

Students like using social networking tools and from the students’ responses in our research, it is evident that 
they accept them as a medium for collaboration and practical learning activities within the higher education 
context. From the survey results presented in tables 1-3 we can conclude that the adoption of the social 
networking tool Elgg for peer-review activity had educational value for students. 

According to the results of our study, we can suggest the following responses to our research questions: 
(1) students’ experience in performing online peer-review activity using the social networking platform Elgg 
were predominantly positive; and (2) the majority of students had a favorable perception of the use of the 
social networking platform Elgg for project peer-review activity. 

There are several limitations of this research. Firstly, the survey questionnaire was completed only by the 
students who chose to do so, so the opinions of other students who also participated in the online peer-review 
activity are not represented in the results. Secondly, 79% of the students in our convenience sample 
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participated in the peer-review activity for the first time so they could not compare it with their prior 
experience in other courses, which may also have affected the results of our case study. Thirdly, the number 
of students in our convenience sample was rather low, which has a negative impact on the statistical 
significance and generalizability of the results of data analyses. 

Nevertheless, we encourage instructors with similar course activities to feel confident about integrating 
social networking tools like Elgg as well as the peer-review activity into their course curriculum taking into 
account both their effectiveness and acceptability among graduate college students. It is our conclusion from 
this and other Web-2.0 related projects that using social networking tools in the educational context 
contributes to the improvement of students’ learning experience, skills development and may also increase 
their motivation for work. 
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ABSTRACT 

Mobile learning is seeing a fast adoption with the increasing availability and affordability of mobile devices such as 
smartphones and tablets. As the creation and consumption of educational multimedia content on mobile devices is also 
increasing fast, educators and mobile learning providers are faced with the challenge to adapt multimedia type 
educational content in order to suit the variety of devices that are used by mobile learners. This paper proposes a solution 
for multimedia profiling that groups mobile devices in classes based on their resolution in order to allow for the creation 
of a reduced number of multimedia clip versions. This solution would support educational multimedia visualization on a 
large set of mobile devices. The paper also makes recommendations for each multimedia profile in terms of audio and 
video encoding settings. 


KEYWORDS 

Mobile learning, multimedia profiles, educational multimedia clips. 


1. INTRODUCTION 

Mobile devices, in particular smartphones and tablets are increasingly used for conducting a multitude of 
online and offline activities, among which mobile learning. According to a recent market research report the 
global smartphone sales have crossed the 1 billion mark in 2013, overtaking feature phone sales in the 
process (CCS Insight, 2013). Moreover, almost 220 million tablets were sold worldwide in 2013, being 
estimated to overtake combined desktop and laptop PC sales by 2015 (Statista, 2014). At the same time, there 
has been a fast growth in multimedia content creation and consumption, with mobile video being estimated to 
increase 14-fold between 2013 and 2018 (Cisco, 2014). 

Mobile and multimedia technologies have also changed radically the online learning landscape. The 
advances in mobile technologies such as improved network speeds, improved processing power, improved 
graphics and higher-resolution displays enable enhanced, more complex mobile learning experiences. As 
mobile devices are gradually becoming the primarily means for accessing the Internet (Meeker, 2013), 
learners are gradually shifting from traditional e-learning to mobile learning (Ambient Insight, 2013). 

Thanks to the latest technologies, mobile learners can easily access educational multimedia content 
anywhere and anytime. Multimedia type educational content has the advantage of providing a rich display of 
information and can be used to further enforce the understanding of difficult concepts through computer 
generated animations, lecture recordings or screencasts. However, a number of challenges such as the 
multitude of mobile devices with different characteristics and the lack of clear multimedia encoding 
recommendations make difficult the adaptation of educational multimedia content based on learner’s device 
characteristics (Moldovan & Muntean, 2011). 

This paper comes to the help of all those involved in multimedia-based mobile learning being them 
educators, educational content creators and/or developers and administrators of mobile learning systems and 
applications. By looking at the current mobile devices market, the paper defines a set of multimedia profiles 
and provides encoding recommendations that enable optimum educational content delivery to a broad range 
of mobile devices. 
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2. ANALYSIS OF MOBILE DEVICE SCREEN RESOLUTIONS 


One of the main challenges in the current mobile learning context, that this paper aims to address through the 
novel multimedia profiling solution, is posed by the multitude of mobile devices with different characteristics 
(e.g., screen resolution) that make difficult educational multimedia clips adaptation based on learner’s device. 

This multi-device screen resolution issue is illustrated in Table 1. The table presents a list of 25 
smartphones and tablets with different screen resolutions that have been recently released on the market by 
various manufacturers and are suitable for mobile learning. Out of the 25 mobile devices, 21 devices were 
released in 2013, 3 devices were released in 2012, and one device (i.e., Apple iPhone 4s) was released in 
October 2011 but it is still on sale as of March 2014 due to its popularity. The table was generated based on 
information from online mobile device specification repositories such as PDAdb 1 and Phone Arena 2 . 

The screen size of the mobile devices listed in Table 1 ranges from 2.8 inches to 10.1 inches. There are 7 
unique aspect ratios across the 26 individual display resolutions, with the most common ones being 16:9 for 
eight resolutions, 4:3 for 6 resolutions, and 5:3 for three resolutions. The aspect ratio represents the 
proportional relationship between the resolution width and the resolution height. The analysis of current 
mobile device screen resolutions includes only resolutions equal and higher to 320x240 pixels. This was 
defined as the baseline resolution for smartphone devices to be used for mobile learning in the M-learning 
Standard (Drinkall & Kneebone, 2012) developed as part of Australia’s National VET (Vocational Education 
and Training) E-learning Strategy. 

While Table 1 provides an idea on the variety of screen resolutions variety of screen resolutions of 
devices that can be used for mobile learning, it provides little indication of the actual usage or popularity of 
different display resolutions. Therefore, to provide a better insight, the popularity of mobile device screen 
resolutions was investigated based on mobile web data traffic statistics from more than 3 million websites 
globally provided by the StatCounter Global Stats web analytics service (StatCounter, 2013). 

Table 1. Common screen resolutions for smartphones or tablets, and their classification based on standard video 
resolution classes used by online multimedia streaming services. 


Resolution Display 

Aspect 

Example of Mobile Device 

Class 

Resolution 

Ratio 

Device Model 

Display Size 

Release Date 

1080p 

2560x1600 

16:10 

Samsung Google Nexus 10 

10.1" 

Nov 2013 


2048x1536 

4:3 

Apple iPad Air 

9.7" 

Nov 2013 


1920x1200 

16:10 

Amazon Kindle Fire HDX 

7.0" 

Oct 2013 


1920x1080 

16:9 

HTC One 

4.7" 

Mar 3013 


1800x1080 

5:3 

Meizu MX3 

5.1" 

Sep 2013 

720p 

1366x768 

16:9 

Samsung ATIV Tab 3 

10.1" 

Aug 2013 


1280x960 

4:3 

LG Optimus VU 3 

5.2" 

Oct 2013 


1280x800 

16:10 

Toshiba Encore 

8.0" 

Nov 2013 


1280x768 

5:3 

BlackBerry Z10 

4.2" 

Jan 2013 


1280x720 

16:9 

Huawei Ascend P6 

4.8" 

Jun 2013 


1136x640 

16:9 

Apple iPhone 5 s 

4.0" 

Sep 2013 

480p 

1024x768 

4:3 

Acer Iconia Al-810 

7.9" 

May 2013 


1024x600 

16:10 

Lenovo IdeaTab A 1000 

7.0" 

Jan 2013 


1024x480 

32:15 

Sony Tablet P 

5.5" 

Mar 2012 


960x640 

3:2 

Apple iPhone 4S 

3.5" 

Oct 2011 


960x540 

16:9 

Samsung Galaxy S4 Mini 

4.3" 

Jul 2013 


854x480 

16:9 

Motorola RAZR D3 

4.0" 

Mar 2013 


800x480 

5:3 

Nokia Lumia 720 

4.3" 

Feb 2013 


720x720 

1:1 

BlackBerry Q5 

3.1" 

Jun 2013 

360p 

640x480 

4:3 

BlackBerry Curve 9220 

2.4" 

Apr 2012 


640x360 

16:9 

Nokia 808 PureView 

4.0" 

Feb 2012 


480x360 

4:3 

BlackBerry 9720 

2.8" 

Aug 2013 

240p 

480x320 

3:2 

Acer Liquid Z3 

3.5" 

Aug 2013 


400x240 

16:9 

LG Wine III 

3.0" 

Oct 2013 


320x240 

4:3 

Nokia Asha 500 

2.8" 

Oct 2013 


1 PDAdb.net - Comprehensive Database of Mobile Device Specifications, http://pdadb.net/. 

2 Phone Arena - Phone News, Reviews and Specs, http://www.phonearena.com/. 
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Figure 1 presents the worldwide market share of different mobile display resolutions in 2013, based on 
mobile data traffic statistics collected between January 2013 and October 2013. The figure shows that in 
terms of actual presence on the market the 480x320 resolution is the most popular accounting for 18.47% of 
the mobile web requests. This is followed by the 320x240 resolution with 10.61% of the global share, the 
1280x720 resolution with 7.5%, the 568x320 3 resolution with 6.62%, and the 800x480 resolution with 
6.39%. 

StatCounter Global Stats 

Top 15 Mobile Device Screen Resolutions in 2013 

480x320 
320x240 
1280x720 
568x320 
800x480 
800x600 
640x360 
960x540 
160x128 
480x360 
400x240 
1920x1080 
1280x800 
133x122 
854x480 

0% 2.5% 5% 7.5% 10% 12.5% 15% 17.5% 20% 

Usage share [%] 

Figure 1. Percentage worldwide market share of the top 15 mobile screen resolutions based on data collected by 
StatCounter Global Stats between January 2013 and October 2013. 



3. RECOMMENDATIONS FOR MULTIMEDIA PROFILING 

In the context of this paper, a multimedia profile is defined as a set of recommended values for video and 
audio encoding parameters such as video resolution, video framerate, video bitrate, audio bitrate, number of 
audio channels, etc. The profiles specify how to create different versions of an educational multimedia clip 
that are suitable to different groups of mobile devices that have similar characteristics (i.e., device classes). 

Ideally, for an optimum quality level each learner should receive a multimedia clip version that fits its 
particular device screen resolution. However, in practice this is not feasible due to the large number of 
versions that would need to be created and stored for each educational multimedia clip. 

The overview of the current mobile landscape, presented in the previous section has revealed that while 
there are a multitude of mobile device screen resolutions currently in use, only a limited number stand apart 
as considerably more popular. Moreover, while there is a trend towards higher screen resolutions, small 
resolutions will continue to be used especially in lower -end devices targeting lower budgets and emerging 
markets. In conclusion, although a broad range of screen resolutions should be covered there is no need to 
cover each and every one. Therefore, the multitude of mobile devices can be grouped in classes based on 
their screen resolution, and a single multimedia profile associated to each class. Five classes of mobile 
devices are proposed in this paper one for each of the standard video resolution profiles: 

• Very Small Screen Resolution Devices (VSRD) - for the 240p video resolution profile 

• Small Screen Resolution Devices (SRD) - for the 360p video resolution profile; 

• Medium Screen Resolution Devices (MRD) - for the 480p video resolution profile; 

• Large Screen Resolution Devices (LRD) - for the 720p video resolution profile; 

• Very Large Screen Resolution Devices (VLRD) - for the 1080p video resolution profile. 


3 This is actually the 1 136x640 resolution of Apple iPhone 5+ smartphones, but is reported as for , but report the 480x320 resolution for 
background compatibility in terms of webpages rendering with older iPhone models. 
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3.1 Video Resolution Recommendations 

For a given educational multimedia clip and a given mobile device, the recommended multimedia clip 
version (i.e., corresponding to a particular multimedia profile), should be selected by considering both the 
clip resolution aspect ratio and the aspect ratio of the device’s screen resolution, as the two can often differ in 
practice. For example, a mobile device with a 640x480 resolution can accommodate the 480p profile 
(640x480 pixels clip resolution) for an educational clip with a 4:3 aspect ratio, but only the 360p profile 
(640x360 clip resolution) for an educational clip with a 16:9 aspect ratio. 

However, in case of multimedia content production the wide 16:9 aspect ratio has become almost 
ubiquitous in recent years, with the increasing adoption of HD (High-Definition) 1280x720, Full HD 
1920x1080, and more recently of Ultra HD e.g., 3840x2160 resolutions. These resolutions are used from 
video cameras integrated with mobile devices to professional video cameras. 

Therefore, the research presented in this paper recommends the usage of the 16:9 video aspect ratio for 
encoding the educational multimedia content. Figure 2 illustrates the recommended video resolutions for 
multimedia profiles associated to the proposed mobile devices classes. As indicated in the figure, the 
resolution approximately doubles in size with every profile. 




1920x1080 

1080p 


1 280x720 

720p 

+ 44% 

854x480 

480p 

+56% 


640x360 

360p 

+ 44% 


427x240 240p 

+56% 




Figure 2. Video resolutions recommended for the multimedia profiles associated to the proposed mobile device classes. 


3.2 Video Framerate Recommendations 

Another important video encoding parameter is the video framerate. The video framerate usually follows 
some predefined standard values and is mainly determined by the equipment used for recording the 
educational multimedia content. However, it can be changed later on during the content editing and 
transcoding phases. Examples of standard framerates that are widely adopted for video recording of 
progressive videos, which is usually the case with Internet videos, include 24fps, 25fps and 30fps. 

The 30 fps value is proposed as the recommended framerate for the multimedia profiles associated to the 
proposed mobile device classes. This value provides excellent perceived quality for multimedia content with 
various dynamicity level (Ou et al., 2011). While the 30 fps framerate is the recommended one, in case of an 
educational multimedia clip that has a smaller framerate (e.g., 24 fps, 25 fps, etc.), the framerate of the 
original clip is maintained for all versions. This is because increasing the framerate to 30 fps would have little 
benefit in terms of user perceived quality. 

3.3 Video Codec Recommendations 

The resolution and framerate settings are independent of the video codec being used for encoding the clips. 
As opposed, for a particular video resolution and frame rate, the video bitrate of a compressed educational 
multimedia clip depends on the video codec being used (some codecs offering better compression for similar 
quality level such as good or excellent), as well as of particular settings specific for the video codec. 
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The main considerations when selecting the video codec are: supported across majority of mobile devices, 
cost and compression quality. Examples of video compression formats that are commonly used nowadays, 
include the standardised H.264/MPEG-4 AVC (Advanced Video Codec) (ITU-T, 2009), as well as its open 
source and royalty free alternatives Google’s VP8 4 5 and Xiph.Org Foundation’ Theora 3 . Their next generation 
successors, namely H.265 HEVC (High Efficient Video Coding) (Sullivan et al. 2012), VP9 (Bankoski et ah, 
2013) and Daala 6 , promise to bring significant performance improvements but they have just been recently 
standardised (i.e., H.265) or are still in development (i.e., VP9 and Daala). 

Although it is subject to licensing royalties, H.264 established itself as the most popular video 
compression format for Internet video delivery (Lawler, 2011), having also playback support on every 
popular mobile platform including Android 7 , Apple iOS 8 , Windows Phone 9 10 and BlackBerry 1 ”. Due to its 
widespread use and high compression quality enabled, the H.264 codec is also recommended by the Flexible 
Learning Advisory Group’s M-learning Standard (Drinkall & Kneebone, 2012). Therefore, the research 
presented in this paper considers H.264 as the recommended video codec for multimedia profiling. 

3.4 Video Bitrate Recommendations 


Various aspects need to be addressed when selecting the video bitrate for encoding multimedia clips in order 
to enable an optimum or level of user-perceived quality (Moldovan et ah, 2013). The bitrate has to be 
appropriate to the resolution and framerate (i.e., higher bitrate for higher resolution), and its selection should 
also consider among others, how the content will be distributed to the learners (e.g., download for local 
playback vs. streaming), the wireless networks speed, etc. 

Table 2 presents the proposed video bitrate range for each multimedia profile associated to the five 
mobile device classes. These values correspond to a 30 fps framerate and the H.264 video codec, and can be 
used for encoding educational multimedia clips streamed over a wireless network. 

Table 2. Video encoding recommendations for the five multimedia profiles when streaming over wireless networks 


Device 

Class 

Video 

Profile 

Resolution 

[pixels] 

Framerate 

[fps] 

Video 

Codec 

Video Bitrate 

[kbps] 

VLRD 

1080p 

1920x1080 



2000-3000 

LRD 

720p 

1280x720 



1500-1800 

MRD 

480p 

854x480 

30 

H.264 

600-1000 

SRD 

360p 

640x360 



350-550 

VSRD 

240p 

427x240 



150-300 


The bitrate intervals are based on guidelines and recommendations provided by Apple (Apple, 2011), 
Adobe (Levkov, 2010) and Wowza (Good et ah, 2011). These companies are big players in the adaptive 
multimedia area having developed commercial solutions that are widely deployed, and they also contributing 
to the MPEG DASH (Dynamic Adaptive Streaming over HTTP) standard. 

Higher bitrates could be used if the educational clips are intended for download and local playback. 
However, these bitrate recommendations are more suitable for educational content delivery over the existing 
wireless networks. A recent study based on real-world usage statistics from 78 mobile providers located in 52 
countries indicated that the average mobile connection speed across these operators was 2.71 Mbps (Akamai, 
2013). Moreover, a number of research studies (e.g., Kennedy et ah, 2010; Moldovan et ah, 2011) that 
investigated the adaptation of the video bitrate in order to support high quality multimedia clips delivered 
over wireless networks to mobile devices have used bitrate values in the range of the recommended intervals 
presented in Table 2. 


4 The WebM Project — WebM VP8 Codec SDK. http://www.webmproject.org/docs/vp8-sdk/. 

5 Theora, video for everyone, http://www.theora.org/. 

6 Xiph.org :: daala video, http://xiph.org/daala/. 

7 Android Supported Media Formats, http://developer.android.com/guide/appendix/media-formats.html. 

8 iOS Media Layer Overview, http://developer.apple.com/library/ios/documentation/miscellaneous/conceptual/ip honeostechoverview/ 
MediaLayer/MediaLayer.html. 

9 Supported codecs for Windows Phone, http://msdn.microsoft.com/en-us/library/windowsphone/develop/ff462087. 

10 BlackBerry media support, http://developer.blackberry.com/devzone/develop/supported_media/. 
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3.5 Audio Encoding Recommendations 

The audio stream usually represents the smaller fraction of the overall multimedia clip. However, this is of 
high importance especially in case of educational clips that provide additional verbal explanations to the 
material presented in the video. 

Since users are more sensitive to changes in the audio quality, as opposed to changes in the video quality 
(Ozer, 2013), this research recommends the audio to be encoded using the same settings for all multimedia 
profiles. The audio sampling frequency is the most important encoding parameter to be maintained constant 
in order to avoid audible pops when switching between the different versions of the clip (Ozer, 2013). A 
typical value for the audio sampling rate is 44.1 KHz. 

The H.264 video codec is often used together with the standardised AAC (Advanced Audio Coding) 
(ISO/IEC, 2006) audio codec. For this particular codec, an audio bitrate of 128 kbps is recommended for all 
multimedia profiles. This value was recommended based on results from subjective listening tests that have 
shown that the AAC codec provides good to excellent user-perceived audio quality levels at bitrates as low as 
64 Kbps (Jiang et al., 2012). 


4. EXPERIMENTAL CASE STUDY 

A subjective study was conducted in order to analyse if the proposed recommendations for multimedia 
profiling provide an excellent user-perceived quality level. To address the variability of the educational 
multimedia content, six educational clips were used in the evaluation case study. The clips are representative 
for the broad spectrum of educational multimedia clips, and correspond to six different categories of 
educational clips that are common nowadays: animations, demos, documentaries, presentations, screencasts 
and slideshows. These were selected from a large number of educational multimedia clips that are available 
on the Internet through iTunes U 11 and YouTube Education 12 multimedia services. Figure 3 presents 
representative frames for the six educational multimedia clips used in the study. More details about the clips 
can be found (Moldovan et al., 2014). 



d) Presentation clip ‘ProjectPlanning’ e) Screencast clip ‘PhotoEditing’ 


f) 


Figure 3. Representative frames for the six educational multimedia clips used in 


Slideshow clip ‘Coralslntro’ 
the subjective case study. 


11 Apple iTunes U, http://www.apple.com/education/ipad/itunes-u. 

12 Education - YouTube, http://www.youtube.com/education. 
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The evaluation study consisted in a number of 60 participants viewing the six educational multimedia 
clips on a mobile device and rating their video quality on a 0-100 continuous scale with the following levels: 
Bad (0-19), Poor (20-39), Fair (40-59), Good (60-79) and Excellent (80-100). A Google Nexus 7 tablet 
device running on Android operating system was used for displaying the multimedia clips used for testing. 
The device has a 7" LED-backlit IPS LCD capacitive touchscreen with a resolution of 1280x800 pixels, a 
Nvidia Tegra 3 1.2 GHz CPU and 1 GB of RAM. The clip playback, quality rating and data recording, was 
done through a purpose build Android app. Standard procedures for multimedia quality assessment as 
recommended by International Telecommunications Union were followed (ITU-T, 2008). 

The educational multimedia clips were encoded for the multimedia profile 720p that is suitable to the 
screen resolution of this tablet device (i.e., H.264 video codec, 1280x720 pixels resolution, 30 fps, 128 kbps). 
Two versions of each multimedia clip were assessed, one at the minimum recommended streaming bitrate of 
1500 kbps and one at the maximum recommended streaming bitrate of 1800 kbps. 

The video quality evaluation results are presented in Figure 4. The results show that both the 1500 kbps 
and the 1800 kbps video bitrate values offer an excellent level of user perceived quality. The average video 
quality as indicated by the Mean Opinion Score (MOS), across the six educational multimedia clips was 
97.82 for the 1800 kbps bitrate and 95.06 for the 1500 kbps. The standard deviations of the subjective MOS 
scores, indicate that for the 1500 kbps bitrate there is a higher variability between the quality ratings of 
individual participants. 



Figure 4. Video quality evaluation results 


5. CONCLUSION 

This paper has proposed a solution for multimedia profiling to address the difficulties in creating educational 
multimedia content for the multitude of mobile devices that can be used for mobile learning. The mobile 
devices are grouped in five different classes based on their screen resolution and a multimedia profile is 
associated to each device class. Based on the features of the latest mobile devices released on the market and 
the multimedia encoding technologies currently available, the paper also proposed encoding setting 
recommendations in terms of video resolution, video framerate, video codec, video bitrate and audio 
encoding. These recommendations can be used by educators, content creators or developers of mobile 
learning applications and services. As these recommendations cover multimedia profiles up to the 
1920xl080p video resolution, they will remain applicable in the future even as more devices with high 
resolution displays will be released. An experimental case study involving 60 participants rating the quality 
of six types of educational multimedia clips on a table device, has confirmed the excellent level of user- 
perceive video quality provided by these recommendations. 
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ABSTRACT 

Advancements in technology and innovations in education allow universities to entertain new ways of teaching and 
learning. Some views of what higher education should look like today include that it be easily accessed by anyone who 
wants to be educated, that it cost less than it currently does, and that there be a significant increase in student engagement, 
experience, and the quality of education. This paper presents quasi-experimental data of how various online tools and 
teaching strategies impact student learning outcomes, satisfaction and engagement. Specific variables impacting social 
presence, affect, etc., where tested to determine their impact on different student outcomes such as grades, feelings of 
isolation, student engagement, and perceived authenticity of course materials. Findings suggest that, despite the literature, 
only some factors had a significant impact on student outcomes and that while some outcomes transferred well online, 
others did not; particularly, peer activities and participation in some course components were hindered online. Considered 
here are students’ experiences with online learning, including hybrid and inverted courses, and teaching strategies that 
help meet challenges in different higher-education learning contexts. 


KEYWORDS 

Online Learning, Pedagogy, Active learning, Student Engagement, Social Presence, Learning Outcomes. 


1. INTRODUCTION 

Advancements in technology and innovations in education allow universities around the world to think up 
new ways of teaching and learning that can sometimes help instructors avoid limitations experienced in 
traditional models of education. Technology drives and enables a lot of the new and different methods of 
online teaching and innovation that we hear so much about in the media, amongst our colleagues, and across 
institutions. Some views of what higher education should look like today include that it be easily accessed 
by anyone who wants to be educated, that it cost less than it currently does, and that there be a significant 
increase in student engagement, experience, and the quality of education. Whether for online courses, hybrid 
courses, or some other learning context, many instructors are wary of using too much technology since it can 
be distracting and they worry that some educational technologies can take away from their teaching and 
students’ learning experience. There is a good argument that one can use technology in ways that brings 
classes together; so for example, using technologies and devices that students are using anyways but embrace 
them in ways that create meaningful interactions rather than distractions, or using them in ways that empower 
both instructors and students, and engages them online or in and out of the classroom. 

It seems obvious that online learning technologies help instructors innovative, but what’s the evidence 
that online learning, or the tools and teaching methods facilitate learning? Presented here is Part-Two of a 
three part quasi -experiment that considers teaching strategies and educational technologies that push learning 
beyond boundaries often found in traditional teaching models. Boundaries considered here include social 
presence, affect, behaviour and cognition. Part-One of this research series, has been published elsewhere 
(Berry & Kushnir, 2013). Data in Part-One compared face-to-face and online teaching and learning; Part- 
Two, adds new data on teaching approaches, strategies and course/curriculum design that focus only on 
online teaching and learning. 

In Part-One, half of the students completed an Introductory Psychology course in a traditional face-to- 
face setting while the other half completed the same course completely online (with the exception of term 
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tests and the final exam to ensure academic integrity). In Part-Two, all students completed the same 
Introductory Psychology course entirely online (again with the exception of term tests and the final exam). 
Added here are the findings from the two additional quasi -experimental groups of online students. One of 
the two groups of students received 30-60 minute lecture videos as part of the online course materials, while 
the other group received the same lecture videos chunked into 5-15 minute lecture video segments with 
embedded quizzes that popped up during the short lecture clips. Specific variables impacting social presence, 
affect, etc., where tested to determine their impact on student outcomes (e.g., grades, feelings of isolation, 
student engagement, feeling like the course materials were authentic, etc.). Results show that, despite the 
literature, only some factors had a significant impact on student outcomes. This research series helps 
researchers, instructors and other education specialists understand factors that impact online teaching and 
learning. It also contributes to our understanding of how specific teaching strategies can impact online 
learning in different higher-ed contexts such as online, inverted, and hybrid courses. 


2. LITERATURE REVIEW 

Enrollment in online education continues to grow at a quicker pace than enrollments overall in higher-ed. As 
many universities and colleges struggle with issues of space, scheduling conflicts and budget cuts, some 
believe that online education offers cost effective alternatives to traditional classroom teaching. In 2010 the 
Sloan Consortium reported that online enrollments were up 17%, compared to 12% the previous year. In 
2011, at least 33% of college students had participated in at least one online course and the majority of these 
students (over 82%) were undergraduates (Allen & Seaman, 2011; Parry, 2010; Salcedo, 2010). 

Online courses can be convenient in higher-ed, mitigating constraints of time and space in traditional 
face-to-face courses, and allowing institutions to offer more courses and effectively meet the growing and 
changing needs of students (Gould, 2003; Macon, 2011). The literature is positive for the most part; some 
argue that it might be more cost effective for institutions to offer online courses since they often require less 
overhead than physical classrooms. Others argue that online courses expand the reach of the institution, 
potentially attracting international students and increasing revenue. Some consider online courses as a way of 
retaining undergraduates and ensuring that they graduate on time, while others see the benefits of allowing 
students to learn at their own pace, the flexibility around studying and working (either part-time or full-time), 
the related savings on commuting, of childcare, etc., (Lei & Govra, 2010; Macon, 2011; Salcedo, 2010; 
Wuensch, Aziz, Ozan, Kishore & Tabrizi, 2008). 

Online learning is not always viewed positively. Some curriculum committees routinely question the 
academic rigor of online courses, express concern about academic integrity, and worry that academic rigor is 
often compromised to facilitate online delivery (Schoenfeld-Tacher, McConnell, & Graham, 2001). Before 
institutions invest in online courses, it is important that there be added value, a positive impact on student 
learning and engagement, and assurances that rigor and academic integrity are maintained. 

Some authors argue that online courses increase the workload of faculty. According to Pallof and Pratt 
(2007), it can take 2 to 3 times more time to prepare and deliver the curriculum for online teaching compared 
to face-to-face teaching. Most of this time is due to the effort required to develop online materials (e.g., 
record/edit lecture videos), manage and upload the resulting large files. Despite the extra work, over 30% of 
faculty report teaching online and this is reportedly increasing (Simson et al., 2006). 

Consistent with Means et al., (2009), for this study, online learning is defined as learning that takes place 
partially or entirely over the internet. This definition excludes purely print-based correspondence education, 
broadcast television or radio, video conferencing, video cassettes, and standalone educational software 
programs that do not have a significant internet based instructional component. 

2.1 Student Engagement and Interaction in Online Courses 

While the literature is generally positive about online interactions, and some authors reporting high levels of 
student engagement and interaction (Schoenfeld-Tacher, McConnell, & Graham, 2001), classroom 
interactions and feelings of community are often the reported benefits of face-to-face courses (Homberg- 
Wright &Wright, 2012). When considering the types of interactions (e.g., using Blooms Taxonomy), some 
report high-level interactions in online instruction compared to face-to-face instruction (Schoenfeld-Tacher et 
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al., 2001). Some students rate interactions online to be about the same as those face-to-face (Allen & 
Seaman, 2011), but students who prefer face-to-face classes, do so because they enjoy the classroom 
interactions (Daymont & Blau, 2008). Quality online courses must incorporate a substantial amount of varied 
interactions (Clark-Ibanez & Scott 2008) and the level of interaction in online courses is often seen as a 
predictor of students’ perceived learning (Rovai & Barnum, 2003). 

2.2 Student Satisfaction and Social Presence in Online Courses 

Student satisfaction in a course is often seen as an indicator of successful learning (Parkhurst et al., 2008; 
York, 2008). The argument being that there is a relationship between student satisfaction and students’ 
perception of the quality of their learning (Piccoli, Ahmad, & Ives, 2001). Substantial and timely interactions 
between the students and instructors can reflect high levels of student satisfaction, and this high level of 
satisfaction can also indicate that teaching methods strongly reflect learning goals and student expectations 
(Moore, 2005). However, the link between student satisfaction and student learning is not clear since students 
might report that they are more satisfied with a course that they perceive as being easy, fun, or less 
demanding; these attributes may not necessarily be linked to real measure of success. Macon (2011) reported 
that undergraduates tend to be more satisfied with face-to-face courses than with online courses, while York 
(2008) found that students were as satisfied with online course work as with face-to-face course work; Fillion 
et al., (2007) also found online students to be more satisfied than face-to-face students. Despite these 
competing findings, it is reasonable to expect that in a classroom where successful learning is evident, then 
student satisfaction will be higher (Driscoll et al., 2012). 

Online students who are more satisfied with their learning experience and satisfied with their instructors 
have been found to have a greater sense of social presence in the course (Lyons, Reysen & Pierce, 2012; 
Richardson & Swan, 2003). There is lots of research that suggests that it is difficult to develop social 
presence online at a level that satisfies students. Instructors have to work hard at creating an online presence 
(Richardson & Swan, 2003; Salcedo, 2010). 

2.3 Student Learning Outcomes in Online Courses 

For the most part, when grades are considered, the literature reports no statistically significant differences in 
learning outcomes between online and face-to-face students; some authors report that online students slightly 
outperform their face-to-face peer in the same course (Beeckman et al., 2008; Beyea et al., 2008; Lim et al., 
2008; Parkhurst et al., 2008; Salcedo, 2010). Few show significantly higher differences in learning outcomes 
for online students, and as mentioned above, academic integrity is often a serious concern (Schoenfeld- 
Tacher et al., 2001). Some authors have found that online learning is at least as effective and robust as face- 
to-face learning (Brownstein et al., 2008), providing the same level of instruction (Carter, Emerson, 2012; 
Driscoll et al., 2010; Russell, 1999). In some cases, online instruction is reported to be less effective than 
face-to-face instruction (Urtel, 2008) with students having difficulty keeping up with the requirements of the 
course (Keramidas, 2012). Also, some authors report evidence that hybrid instruction (i.e., combining online 
and face-to-face elements) has a greater advantage relative to purely face-to-face or purely online instruction 
(Means et al., 2010). 


3. RESEARCH RATIONALE 

This study evaluated the impact of various teaching strategies and online tools (e.g., the use of lecture videos, 
quizzes with rapid feedback, peer-to-peer activities, and online assignments) on the following factors; 

1 . Student engagement, interaction and feelings of isolation 

2. Satisfaction of learning experience, social presence (and how well students felt they got to know the 
instructor), and authenticity of course materials 

3. Student learning outcomes 

We set out to investigate what are students’ experiences online, if there are variables or teaching 
strategies that impact the factors listed above, and if certain strategies correlate to better outcomes. 
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4. DESIGN OF THE STUDY 

4.1 Participants and Description of the Study 

A total of 60 students enrolled in an introductory psychology course at an urban university participated in 
Part-One of this study. About half of those participants were enrolled in a face-to-face section (n=31; section 
1, Part-One ) and the other half in an online section (n=29; section 2, Part-One). The students were self- 
selected and chose in which section of the course they enrolled; they were only restricted from switching 
between sections after final date to register in the course (a date set by the University Registrar’s Office). A 
total of 52 students participated in Part-Two of this research series. Similar to the first, about half of the 
participants were enrolled in one section (n=25, students received 30-60 minute lecture videos, and all course 
messages delivered using the text announcement tool in the institution’s learning management system; 
section 1, Part-Two ) and the other half of students were in the other section (n=27, students received 5-15 
minute lecture clips with embedded videos, and course messages delivered via a video messaging tool to 
increase social presence; section 2, Part-Two). As was the case in Part-One, students were self-selected and 
only restricted from switching sections after the final registration date (though students in this second part of 
the study had no reason to move between sections since they were unaware of any explicit differences 
between the groups; as far as they were concerned there were just two online sections of the course available 
concurrently). Across all groups, there were no significant differences between first year university average, 
cumulative average, and high school entrance average. All online students received a “presence” video from 
the instructor as well as all of the lecture material online. Students came to class (physically) only 3 times to 
complete the requirements of the course (for two term tests and a final exam). The face-to-face group (section 
1, Part-One) met for 3 hours, 2 times per week for lectures during a compressed summer semester of 6 
weeks. A key component in the face-face section was peer activities, which were facilitated with student 
response system (clickers). This particular course component was replicated in the online groups by using 
online quizzes and a discussion board in the institution’s learning management system. Initially, break out 
rooms in Adobe Connect were used to facilitate synchronous peer activities but this had to be abandoned due 
to a problem with the tool and therefore Adobe Connect was only used for synchronous online office hours. 
Students received the same course content and course components in all sections of the courses delivered by 
the same instructor (i.e., Summer 2012 and Summer 2013). 

Online student experience was assessed using a half-way checking in survey, while all students received 
an end-of-term survey and other learning outcome comparisons included 6 quizzes worth 10% of the final 
grade, 2 term tests each worth 20% of the final grade, a final exam worth 30%, an assignment worth 10%, 
peer activities and participation worth 10%, and overall final course grades for all groups were compared). 

4.2 Analyses 

Analysis of the data included independent /-Tests to measure any differences between final grades for the 
online and face-to-face students in Part-One, and between the two online groups in Part-Two. ANOVAs 
were calculated to measure any differences between all course grades across the four groups in both parts of 
the research series. Qualitative analyses of the open-ended survey questions included response frequencies 
of the survey questions across the groups and weighted word lists that were calculated and puzzled out into a 
word clouds that were generated from the students’ text answers. The word clouds represented a summary of 
the text that students wrote in their open-ended answers. A user generated word cloud visualizes information 
that is related to a specific survey question and, in essence, it depicts visually, the frequency of specific topics 
that students write about in their answers. The importance (or frequency) of specific words is displayed using 
font size (as in the example below), font colour, or some other attribute (see Bateman et al., 2008 for an 
overview of word/tag clouds). 


5. RESULTS AND DISCUSSION 

5.1 Student Engagement, Interaction and Feelings of Isolation 

Class interactions and feelings of community are some of the reported benefits of face-to-face courses 
(Homberg-Wright &Wright, 2012) and in some cases, students who prefer the face-to-face classes do so 
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because they like interacting with the instructor and their classmates (Daymont & Blau, 2008). This suggests 
that online environments should provide lots of opportunities for engagement and interaction. In our study, 
students were asked to indicate in which part of the course they felt most engaged; as indicated in Figure 1, 
students reported that the peer activities and the Science Meets Art project (a course assignment that was 
shared amongst classmates) provided the most engagement. The demonstration videos as well as the lecture 
podcasts also contributed to student engagement. Online students were also asked whether they wanted more 
interactions with their classmates, and if so, what sort of interactions they wanted. Interestingly, the request 
for face-to-face interactions and study groups came up frequently in students’ responses (Figure 2). This 
provides support for a flipped or inverted classroom where the lectures and usual in class material are placed 
online for students to access outside of class, and where class time can be used for activities that would 
normally be done at home such as homework and assignments. This gives students and instructors the 
opportunity to have engaging, interactive sessions such as collaborative work and in-class activities that 
focus on higher level cognitive activities (Bull el al., 2012; Brunsell & Horejsi, 2013; Milman, 2012). 
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Figure 1 . What part of the course or course activities helped students to feel most engaged 
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Figure 2. Students indicated that they would have liked more interactions with classmates; 
these are the sorts of interactions that they wanted to have (online students only) 

When asked if they felt isolated, online students in section 2, Part-Two (the group with 5-15 minute 
lecture clips with embedded videos, and course messages delivered via a video messaging tool to increase 
social presence) reported the least isolation (see Figure 3). This suggests that these students feel more 
included and connected in the course compared to the other online groups who seem lonelier. 

Did you feel isolated in the course? 
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Figure 3. Online students’ report of feeling isolated in the course 


5.2 Student Satisfaction, Social Presence and Authenticity 

We also asked students about what they found most satisfying about the course. Overwhelming, across all 
groups students reported being satisfied with their experience and their learning; figure 4 shows what 
students found most satisfying. As reported earlier, online students who are more satisfied with their learning 
experience and satisfied with their instructors have been found to have a greater sense of social presence in a 
course (Lyons et al., 2012; Richardson & Swan, 2003) indicated here (in Figures 5 & 6) by how well they got 
know the instructor, and what course factors helped them feel like they knew the instructor (Figure 7). We 
also found that this had an impact on students’ perceived authenticity of the course materials, and how real or 
artificial the course felt to students (Figure 8). 
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5.3 Learning Outcomes 


Students were asked what aspects of the course contributed most to their learning (Figure 9). While students 
reported that the interactive and active components contributed most to their learning, there were no 
significant differences between the groups on their grades for any of the course component ( i.e 6 quizzes, 2 
term tests, final exam, course assignment, peer activities, course participation, or final course grade). 
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Figure 9. What students believed helped most in their learning 
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6. CONCLUSION AND RECOMMENDATIONS 

We set out to evaluate the impact of various online tools and teaching strategies (podcasts, online 
assignments, quizzes, rapid feedback, discussions, and peer activities) on student learning outcomes 
engagement and satisfaction of their learning experience. We discovered that students had similar 
experiences across the different groups supporting the argument that online instruction can provide at least 
the same level of instruction and satisfaction as face instruction (Driscoll et al., 2010; Russell, 1999). In our 
study, we discovered that the teaching strategies that we chose actually influenced the teaching tools, which 
in turn, had some influence on our strategies. This is important that online courses be built on sound 
pedagogical principles in order to facilitate meaningful and successful learning. We took the necessary steps 
to design the online course around our teaching and learning goals that supported instructional needs and 
student learning outcomes, and not around the teaching tools that happen to be available. As more and more 
university instructors look to educational media and technologies to help engage students and enrich learning 
environments in different learning contexts, it will be helpful if future research focuses on the use of different 
learning contexts and online innovations to facilitate different teaching and learning methods, both in and out 
of the classroom. Advancements in technology and innovations in education allow universities to think up 
new ways of teaching and learning. It is in understanding the pedagogy behind the technology that will get 
us further along in understanding how to best implement educational media and technologies to enrich these 
learning environments. 
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ABSTRACT 

With the advancement of infonnation and communication technology, collaboration and knowledge sharing through 
technology is facilitated which enhances the learning process and improves the learning efficiency. The purpose of this 
paper is to review the methods of assessment and strategy of collaboration and knowledge sharing in a computer course, 
and find out how these approaches improve students’ learning. The computer course is aimed at helping students to 
master knowledge, building their operational capability and equipping students with analytical skills in using and 
operating computers. While traditional evaluation model restrains students’ learning interests and innovation, e-leaming 
is introduced to support course delivery and enhance collaboration and knowledge sharing among students. Different 
assessment methods are discussed with respect to the impact of collaboration and knowledge sharing on students’ 
learning. The assessment methods are carefully analysed and evaluated in order to find out how the collaboration and 
sharing of knowledge among students improves learning experience. Suggestions for improvement are made to increase 
the quality and quantity of collaboration and knowledge sharing in and beyond the classroom setting. 


KEYWORDS 

Knowledge sharing, assessment method, computer course. 


1. INTRODUCTION 

During the 21st century, the rapid development of Information and Communication Technology (ICT) has a 
strong impact on all walks of life all over the world (Weide, 2012). In Hong Kong, the wide adoption of ICT 
systems in both public and private sectors assures Hong Kong as one of the world’s most advanced digital 
cities. In order to equip tertiary students with adequate skills in computer applications and increase their 
competitiveness in the job market, the study of information technology has probably become one of the 
compulsory modules in the curriculum offered at higher education institutions in Hong Kong. 

Hong Kong Community College (HKCC) of the Hong Kong Polytechnic University (PolyU) is a self- 
financed post-secondary institution which offers Associate Degree (AD) and Higher Diploma (HD) 
programmes. HKCC offers IT course for all students to acquire basic computer knowledge. For instance, the 
course “Applied Computing” is a generic course for all year one students under the associate degree scheme 
in science and technology. The course objectives are outlining information technology and its applications, 
developing end-user computing skills and integrating end-user computing techniques into business 
applications. The course is aimed at helping students to master knowledge, building their operational 
capability and equipping students with analytical skills in using and operating computers. While traditional 
evaluation model restrains students’ learning interests and innovation, e-learning is introduced to support 
course delivery and enhance collaboration and knowledge sharing among students. 

In this paper, we are going to study the methods of assessment of the course “Applied Computing” and 
find out how students’ engagement in collaboration and knowledge sharing can improve learning. Through 
analysing the methods of assessment, strategies for improving students’ learning and classroom instruction 
will be formulated. This paper will start with a literature review of knowledge sharing and principle of 
assessment for learning. The methods of assessment of the course will be discussed, with a review of how 
knowledge sharing among students is conducted to improve learning and reach the purpose of Assessment 
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for Learning (AfL) (Berry, 2008). Suggestions on increasing the quality and quantity of knowledge sharing in 
the course will be stated and some concluding remarks will be given at the end. 


2. LITERATURE REVIEW 

2.1 Knowledge Sharing 

Knowledge sharing, in the meantime, is a core component of and a challenge issue for knowledge 
management. Knowledge sharing takes place when an individual provides information and know-how to help 
others and collaborate with others to solve problems, develop new ideas, or implement policies and 
procedures (Wang & Noe, 2010). Knowledge sharing can occur via face-to-face communications or written 
correspondence through networking with other experts, or documenting, organising and capturing knowledge 
for others. The term differs from knowledge transfer and knowledge exchange, where knowledge transfer 
refers to the movement of knowledge between the knowledge source and the recipient, and knowledge 
exchange includes both knowledge sharing and knowledge seeking. 

Technologies are crucial to the sharing of information, which provide an efficient and automated means 
to track data over time, interact with others, post information, and share discoveries (Petrides & Nodine, 
2003). With the advent of powerful and sophisticated hardware and software tools, the collection, storage, 
and distribution of knowledge can be better supported and easily performed with a few clicks. At the same 
time, it is important to look beyond technical capabilities and focus on other factors that influence knowledge 
sharing, for example, organisational context, interpersonal and team characteristics, cultural characteristics, 
individual characteristics, and motivational factors (Wang & Noe, 2010). 

In the field of education, traditional education model is arranged such that teacher always acts as a 
repository and transmits knowledge to students in classrooms. The incorporation of state-of-the-art 
technology assures learning is moving towards practice-based, interactive -based learning. The latest Web 2.0 
technologies offer platform for creating collaborative learning environments which foster meaningful 
learning (O’Reilly, 2007) and allow knowledge sharing on a globe scale (Tarik & Karim, 2011). Web 2.0 
helps students not only in receiving information, but stimulates them in brainstorming, collaborating and 
learning through knowledge sharing. In higher education institutions, learning management system is 
sometimes adopted as a knowledge management system to elicit and support the sharing of knowledge 
among students. By means of an appropriate mix of e-learning and traditional classroom learning, blended 
learning extends the student learning continuum through the e-learning components and activities outside 
classroom (Lam et al., 2011; Lam, Hung, Chan, Yan, & Woo, 2011). By implementing Web 2.0 technologies 
and appropriate strategy of knowledge sharing, students’ collaboration in and beyond the classroom setting 
can be enhanced and strengthened. As a consequence, learning is better supported in the new era. 

2.2 Assessment for Learning 

The concept of assessment for learning gives a general direction of where assessment should go. Ten 
assessment principles for assessment for learning have been developed, presented within the framework of 
Assessment of Learning (AoL), Assessment for Learning (AfL), and Assessment as Learning (AaL), as staed 
in Berry, 2008. The core features of AoL, AfL, and AaL are on product of learning, process for learning, and 
learner taking control respectively, which are closely related to learning in numerous ways. Teacher, 
students, and system can all contribute to making assessment effective. The ten guiding principles listed 
below will be of help when actions are being considered. 

1 . Aligning assessment to teaching and learning 

2. Exploring the use of multidimensional assessment methods 

3. Selecting those assessment methods which are susceptible to learning 

4. Considering drawing on joint efforts among colleagues 

5. Assessing students continuously throughout the learning processes 

6. Allowing students to take part in the assessment process 

7. Using assessment to uncover students’ learning 
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8. Making marking criteria accessible for students 

9. Providing feedback to facilitate students’ learning 

10. Analysing and reporting students’ results 

The fundamental principle of assessment for learning illustrates how assessment and learning are closely 
related. Assessment plans should be carefully designed to promote, induce and reinforce learning. Students 
are actively involved in their own learning, with the ability to assess themselves and understand how to 
improve. Teachers are responsible for monitoring the assessment results to give timely feedback and adjust 
teaching where appropriate. Within the parameters of assessment for learning, students’ involvement in the 
assessment activities is taken seriously, as they are the key players of learning. Teaching, learning and 
assessment have to come together and work together if we are to raise students’ standards of achievement 
and help students achieve deep learning (Berry, 2008). In the next section, the methods of assessment and 
strategy of collaboration and knowledge sharing are discussed and evaluated based on the abovementioned 
assessment principles. 


3. THE METHODS OF ASSESSMENT AND STRATEGY OF 
KNOWLEDGE SHARING IN COMPUTER COURSE 

The course “Applied Computing’’ is a generic course for all year one students under the associate degree 
scheme in science and technology. This year, around 900 students enrolled in the course and were placed in 
one of the nine classes being taught by five teachers. The course delivery was identical for all classes, with 
the same set of course materials and methods of assessment. The assessment components included two 
individual assignments, a group project, and an examination. Based on one of the classes of this course, this 
paper will examine the impact of collaboration and knowledge sharing on individual assignments and group 
project, and discuss how these methods of assessment help students improve learning. 

3.1 Individual Assignments 

The course included two individual assignments, i.e., a take-home programming assignment and an in-class 
practical exercise. For the programming assignment, students were asked to write computer programs in 
order to solve a given problem. For the practical exercise, students were asked to answer 15 multiple-choice 
questions during class. The course made use of an e-learning platform and an automated assessment system 
to construct a better learning environment for students (Berry, 2003). 

3.1.1 e-Learning Platform 

To engage students in the learning process, e-learning is adopted to improve their learning experience. An e- 
learning platform, which is a Moodle-based learning management system, is used for managing course 
materials and course activities. In this study, students were provided with lecture notes and reading materials 
in the course website. By making use of the assignment and quiz functions, both programming assignment 
and practical exercise were made available online. Students could make use of the discussion forum to post 
and discuss questions with teacher and other students. With the grading function, teacher could provide 
assessment feedback and grading to students easily, and monitor students’ learning progress systematically. 
Other built-in functions of the e-learning platform, e.g., announcement, calendar, etc., were also applied in 
the course setting. 

In the beginning, most students did not have much knowledge of writing computer programs and came up 
with lots of questions while working on the programming assignment. As such, students were encouraged to 
fully utilise the online discussion forum to post and discuss questions with other students. This helped 
students to share their knowledge with peers in different aspects, thus achieving active knowledge sharing 
among students. Besides, as students might have similar questions in mind, they could discuss their 
difficulties in the forum and work collaboratively to solve their problems. Teacher could monitor and guide 
the online discussion, as well as address their difficulties in class if needed. In brief, there were a number of 
advantages to use an e-learning platform (Slack, Beer, Armitt, & Green, 2003). 
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- Efficient. With e-learning platform, students can discuss problems or difficulties in the online forum 
which helps them to have more understanding on related topic through discussion and develop a habit 
of communicating with others through network. Thus, learning is no longer restricted to the classroom 
and contact hours, but knowledge sharing can occur anywhere and anytime. When students start 
discussion in the forum, teacher can monitor their discussion and offer guidance and timely feedback 
to facilitate their learning where appropriate. In this way, teacher can make sure students’ learning is 
on the right track throughout the learning process, resulting in improved efficiency and effectiveness 
of learning. Teacher can also assess students’ learning progress in a continuous manner and make 
necessary adjustment accordingly. That is, principle 5 and principle 9 of AfL are achieved. 

- Collaborative. Students can learn from each other through discussion in the online forum. Students 
share their own knowledge and experience in the discussion. For programming exercise, one may 
come across different problems under different situations. Through discussion, highly capable 
students can help less capable ones in problem solving, and learn from their mistakes and have a 
deeper understanding of the topic simultaneously. This improves learning of all engaged students of 
the course and creates an inclusive learning environment. 

- Consistent. Teacher can post announcement and make amendment on assignment conveniently and 
consistently via e-learning platform. For programming exercise, it is often necessary to make 
clarification and amendment from time to time. Meanwhile, assessment criteria can be posted and 
made accessible by students. This fulfills principle 8 of AfL. As a result, students understand clearly 
what is expected from the assessment task and are able to learn through assignment with clearer 
learning outcomes (Rust, Price, & O’Donovan, 2003). 

3.1.2 Automated Assessment System 

Computer technology can be used for assessment purposes at various levels ranging from management of 
assessment information to a fully automated assessment system. In this study, students submitted their 
programming assignment online where all submissions were marked by an automated assessment system. 
The assessment results were generated with statistical analysis for both students and teacher. With the 
automated assessment system, assessment information could be retrieved and presented in different ways to 
meet the needs of students, teachers, course organisers and external examiners. With the computer-based 
assessment, students’ learning and knowledge sharing among students were improved and enhanced in the 
following ways. 

- Prompt Feedback and Report. The computer-based assessment is programmed to provide assignment 
details to students, grade the submitted assignments, and return immediate results to corresponding 
students and teacher. In Hong Kong, students are result-oriented and urged to get assessment feedback 
within a few days. Through the computer-based assessment, students can get full report of the 
assessment within a short time period. For the programming assignment, teacher needs to test 
different cases during marking in order to reflect different errors in the programs. The computer-based 
assessment can provide correct answers immediately while identifying errors made by students. With 
the full marking report, students have a full picture of their performance and improvement can be 
made based on the feedback. This helps students to learn from their mistakes and achieve deeper 
learning ultimately. In general, an assignment alone is merely not enough or complete in the learning 
process. Without reflecting upon the assignment it may be forgotten quickly or its learning potential 
may be lost (Brady & Kennedy, 2005). As students may forget what they have done in an assignment 
after a period of time, a marking report provided promptly allows students to recall details of the 
assignment and learn from their mistakes more efficiently. 

- Adjust Teaching Strategy. With automated assessment, assessment against each submission will be 
consistent, including scope, content, and presentation of the assessment report. It facilitates the setup 
of a comprehensive assessment. Teacher can further elaborate analysis of the assessment results and 
find out common mistakes made by students. Teaching strategies can be adjusted accordingly which 
improve students’ learning and learning efficiency. Teacher can align learning outcomes with 
assessment in order to reinforce teaching and learning. Besides, teacher can upload more reference 
materials related to the assignment to enhance knowledge sharing of related topics with students. 
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3.2 Group Project 

The course included one group project which students were asked to develop a webpage with a particular 
topic. Our analysis is based on how students learn through group work and the peer and self-assessment by 
means of computer technology. 

3.2.1 Learn through Group Work 

Group project is widely adopted as a teaching and learning activity. It provides useful information about 
students’ understanding and knowledge of particular learning areas, their abilities to apply knowledge in 
particular investigations, and their abilities to communicate subject-specific information clearly (Chapman & 
King, 2005). Through this group project, students acquired understanding and technique by discussing with 
their group members. They shared knowledge, built up team spirit, and worked with each other to complete 
the group project enthusiastically. Teamwork is one of the key factors for knowledge sharing. With the ICT 
applications, group project supports students to learn more efficiently. 

- Independent Learning. AfL is a source of motivation and a key element in the development of 
independent learner. For group project, students work in a team environment to discuss and set their 
learning goals and get a clear direction at first, which help them to understand the criteria clearly and 
answer the questions "Am I getting it?" and "How am I doing?" Besides, students need to search for 
reference materials to develop the project which helps them to learn a particular topic deeply and 
independently. As students need to apply the theory learnt in the project, this helps them to revise the 
course content. Technology is considered to be an important contribution since students can 
collaborate even without physical presence. In addition, technology can act as a facilitator to 
encourage and support knowledge sharing by making knowledge sharing easier and more effective 
(Riege, 2005). Team members can share their views and materials collected via e-learning platform 
which help them to perform their work more efficiently. Through the group work, students are trained 
to be active, independent learners whom take ownership of their learning. 

- Leadership. Leadership development is also an important factor for knowledge sharing. Team leader 
facilitates knowledge sharing and engenders trust among group members which contribute to team 
effectiveness. Students working in a group need to build up team spirit and communicate with each 
other. Through discussion, each student takes turn to present their ideas and comment on others’ 
work. The project can be sub-divided into several tasks such that each student leads one part of the 
project and works together for different parts of the project. Through the group work, students 
develop leadership skills and learn to work with each other to accomplish the group project. 

- Self-reflection and Peer Evaluation. Group project can provide students with a valuable learning 
experience by combining talents of group members through contributing knowledge and ideas. 
Engagement and commitment are required to make the most of the contributions of all students. Self- 
reflection helps students to develop self-discipline and carry out self-evaluation, as well as identify 
strengths and seek assistance for overcoming weaknesses. Besides, peer assessment is not merely a 
means for awarding marks, but allows students to give valuable feedback to their group members to 
facilitate collaborative learning. It adds a valuable dimension to learning, i.e., the opportunity to talk, 
discuss, explain and challenge each other, which provides opportunities for students to evaluate and 
develop objectivity in making judgments about their own work and the work of others. In general, 
peer assessment helps students to develop self-regulation, which promotes independence in learning 
and assures students take some significant responsibility for setting their own project goals and 
evaluating progress against these goals (Nicol, 2010). 


4. EVALUATION AND SUGGESTIONS FOR IMPROVEMENT 

The methods of assessment of the course “Applied Computing” were designed to facilitate the sharing of 
knowledge among students and improve their learning experience. To evaluate the overall arrangement, 
interviews were conducted to obtain students’ feedback and identify areas for improvement. 
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4.1 Results of Interviews with Students 

In this study, semi-structured interviews were conducted with three students who studied this course. A set of 
questions were formulated to obtain students’ feedback on assessment practices and how these assessments 
affect their learning experience. 

The questions were classified into three categories, which included general questions, strengths and 
weaknesses of the methods of assessment. For general questions, we aimed at understanding students’ 
general impression on knowledge sharing through e-learning platform. After that, some questions were 
prepared for collecting students’ experience of strengths and weaknesses of the methods of assessment. 
Transcripts of the three interviews were consolidated and coded, which is shown in Table 1. Through 
analysing the interview transcripts and gathering research findings from similar studies, suggestions for 
improvement are summarised in the next section. 

Table. 1. Coded Transcripts of the Semi -Structured Interviews 

Category Transcripts Coding 


General “We can download course materials easily.” (Henry) e-Learning helps students to manage their 

study with ease. 

“We can get the notes whenever we want, and can reprint 
the notes easily.” (Alvin) 

“The assessment criteria can be retrieved easily via e- 
leaming platform.” (Simon) 

“Discussion with classmates through e-leaming platform e-Learning helps students to 
helps us to better understand the assignment.” (Henry) communicate with teacher and peers. 

“Teacher responses to our questions quickly.” (Simon) 


“Teacher would clarify the uncertain points quickly via e- Timely feedback is useful for improving 
learning platform or address them during class, which students’ learning, 

helps a lot.” (Simon) 

“More feedbacks are received through e-learning 
platform.” (Alvin) 

“Students can improve their project after obtaining 
feedback from teacher and other students.” (Henry) 


Strengths “Getting the assessment results of assignment timely helps Automated assessment system helps 

us to better understand our learning.” (Alvin) students to improve learning. 


“Group project helps us to learn how to work together and Group work introduces another learning 
work out the solution as a team.” (Henry) culture and practice and encourages 

collaboration. 


“Sometimes we can find out the solution through Group work builds self-confidence and 

discussion.” (Simon) improves self-esteem. 

“We acquire computer skills through project 
implementation.” (Alvin) 


“With self-reflection, we know what we have learnt in a Peer and self-assessment engages 
better sense.” (Henry) students in reflecting upon their learning 

process. 

“Peer evaluation in group project is important to reflect our 
contributions.” (Simon) 
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Weaknesses “Some classmates may not join online discussion as they Participation in online discussion is not 

are not used to the e-leaming mode.” (Henry) sufficient. 

“Teacher may not answer all questions in e-learning 
platform.” (Alvin) 


“Sometimes we may not have enough support on the use of Support on e-learning is to be 
e-leaming technology.” (Henry) strengthened. 


“Assessment results of assignment are expected to include Automated assessment is to be enhanced 
more details.” (Alvin) with more feedback. 


4.2 Suggestions for Improvement 

4.2.1 Promote Participation in Online Discussion via e-Learning Platform 

The e-learning platform is adopted as a communication channel outside classroom. Students make use of the 
discussion forum to post and discuss questions with other students. However, it is found that some students 
are not aware or do not check the discussion forum regularly. To promote participation in online discussion, 
teacher can highlight some of the posts in discussion forum dining class and encourage students to check for 
details via e-learning platform. In addition, the sharing of knowledge through online discussion can be 
included as a form of assessment activities, where such assessment criteria can be based on quality (content) 
and quantity (number of posts) of knowledge sharing. 

4.2.2 Provide More Feedback on Automated Assessment 

Automated assessment is efficient and time saving, while ensuring consistency in the comments. However, 
the reports generated by automated assessment system may not be sufficient for students to understand their 
performance and learn from their mistakes. Teacher can clarify the assessment criteria and provide more 
feedback on automated assessment during class. Besides, related reference materials can be shared with 
students via e-learning platform. 

4.2.3 Implement Self-Assessment in Assessing Individual Assignments 

Self-assessment allows students to become more engaged in their own learning by reflecting on the quality of 
their work (Taylor & Nolen, 2005). Practice quizzes and written assignments are both examples of self- 
assessment. Implementing self-assessment helps students to develop self-regulation and evaluate their 
learning progress. It encourages students to identify strengths and seek assistance for overcoming 
weaknesses. By allowing multiple attempts to practice quiz and/or written assignment followed by additional 
instruction, benchmark measures can be identified for students to measure progress (Mehta & Xavier, 
2007/8). 


5. CONCLUSION 

This paper has reviewed the methods of assessment of a computer course with the incorporation of computer 
technology and strategy of collaboration and knowledge sharing. e-Learning platform provides a channel for 
online discussion among students which reinforces their learning and promotes collaborative learning. 
Automated assessment assures consistency in the comments such that students can receive prompt feedback. 
Group project allows students to work in a team environment to study the project requirement, research on 
the topic, discuss on the findings and accomplish the project. Peer and self-assessment engages students in 
reflecting their own learning process and evaluating the work of others. Throughout the course delivery, 
knowledge sharing is incorporated into assessment in order to improve students’ learning and achieve deeper 
learning. 
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To promote and enhance knowledge sharing in the course, a number of suggestions have been stated. In 
general, the scope of knowledge sharing can be expanded from students of the same class to all students of 
the course, and to a wider community. With the latest Web 2.0 technologies and appropriate strategy of 
knowledge sharing, quality and quantity of knowledge sharing in and beyond the classroom setting can be 
significantly increased. In other words, learning can be better supported and active learners can be benefited 
in the new era. 
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ABSTRACT 

Recent research has shown that educational games positively motivate learning. However, there is a little evidence that 
they can trigger learning to a large extent if the game-play is supported by additional activities. We aim to support 
educational games development with an Agent-Based Technology (ABT) by using intelligent pedagogical agents that can 
intervene to offer hints, assistance and suggestions when the learner is lacking knowledge, but does not intervene 
otherwise, so as not to interrupt game flow. In this paper we describe the possibilities of using pedagogical agents to infer 
learner’s motivation and emotional state as they allow communication and interaction in a digital learning environment. 
Our approach emphasizes on improving pedagogical agent interactivity: from pedagogical agent techniques to Tutor and 
Tutee Agents’ techniques to create a high social and collaborative Digital Educational Game (DEG) environment. 


KEYWORDS 

Educational games, motivation, emotional state, agent-based technology, tutor agent, tutee agent 


1. INTRODUCTION 

Recently, one direction of technology enhanced learning that has been attracting increased attention is the 
study of educational agents, which are human-like virtual characters that play specific educational roles in 
their interaction with students during learning. With this development we have deem it necessary to 
incorporate the agent-based technologies in educational games development in order to improve its’ learning 
goals that they were originally meant for. Agent-based technologies can help students develop a positive 
relationship with the educational agent, and further enhance their motivation in terms of attention, relevance, 
and satisfaction. 

There are two reasons for embracing agent-based technologies in educational games development. First, 
the educational agents are regarded as friendly interfaces that can facilitate interaction with students, leading 
to enhanced motivation and the perception of improved ease and comfort. 

Such features can also be applied to benefit student learning so that they can become engaged in the 
learning environment (Gulz, 2005). Second, a number of advantages have been reported to learning with the 
support of different types of educational agents on specific aspects, such as exploration (Hook et al., 2000), 
reflection and articulation (Tholander et al., 1999), communication (Johnson et al., 2000), and negotiation 
(Bull, 2004). This might be because pedagogical agents can serve as learning companions offering virtual 
participation in a DEG. By doing so, students are encouraged to interact with the learning agent (Pedagogical 
Agent) in a social context and experience an enjoyable learning process, which, in turn, results in the 
aforementioned learning advantages. 


1 This work was partially funded by the Ministry of Education and Science of Russian Federation within the basic part (task 2586 of 

project 2014/16). 
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In this paper we further describe the relevance of adopting an agent-based technology in educational 
game development. First we provide some background on agent-based technologies in learning systems and 
related technologies in educational games. Next, we would give a brief overview of pedagogical agent 
intervention and we would discuss the features of improving single pedagogical agent techniques to Tutor 
and Tutee Agents’ techniques, and they implementation in adaptive educational games. 


2. AGENT-BASED TECHNOLOGIES IN LEARNING SYSTEMS 

When we talk about agent-based technologies in learning system we are referring to educational agents. 
Educational agents are human-like computer simulated characters that are designed to improve student 
learning in an individual environment through virtual participants (Chou, Chan, & Lin, 2003). In recent time 
educational agents have tremendously make valuable remarks in cognitive and affective aspects in digital 
learning environment. This is because the influence of the educational agents on student learning can be 
expanded by taking the affective qualities of the virtual character into account in the design (de Vicente & 
Pain, 2002). 

Educational agents are good tools worth considering when designing a learning system. They are tools 
that exist with external goals, there are easy to use, in order to influence the user’s external goal. For example 
some good educational games are difficult to play, as a result of increased challenges provided to the player. 
Whereas typical interactive systems like educational agents, system build for human learning should be easy 
to use. The external goal is for the user to learn how to perform a given task, so the system should make the 
process of learning how to accomplish that task easy — the process of learning the task, not the task (E. Sklar 
2003). 

Current developments on Intelligent Tutoring Systems (ITS) also consider a co-operative approach 
between the learner and the system. Many research groups created environment-using simulation where the 
teaching-learning process is simulated by a set of agents in order to have interaction among them and allow 
us to observe the dynamics changes that occur during the interaction process. One example is presented by 
Moussalle (Moussale 1996), where all the agents must be opened to change strategies and 
believes/knowledge about how to solve problems. 

One major significant about agent-based technologies in a learning system is that it brings about social 
learning environment which mean collaborative work. Mitssuru said: one of the major educational 
significance of the collaborative learning is to enhance the participant’s motivation to awaken mature 
reflections on their own understanding and externalize the result of it (Mitssuru 1997). This kind of 
environment can be created when one or more agents are simulated in the computer: Tutor and Tutee. As a 
tutor the agent intervene by giving hind, the right solution at the appropriate time etc. And the Tutee Agent 
provides collaboration support and tries to test the student’s self-confidence and his knowledge. 


3. ADOPTING AGENT-BASED TECHNOLOGIES IN EDUCATIONAL 
GAMES 

3.1 Pedagogical Agents in Educational Games 

Pedagogical agents in educational games facilitate the pedagogical assignment task in a learning session 
context. The idea of an agent-based approach is to represent the pedagogical knowledges and its use in a 
tutoring context. The fundamental reason for adopting agent-based technologies in educational games as a 
tutoring knowledge element is their capabilities of communicating and interacting. 

According to Vassileva (Vassileva 1997), an agent must act in a world populated by other agents, because 
many agent’s goals require the help of another agent. In this way, relationships among agents can be viewed 
as another kind of resources for achieving goals in a game playing environment. 
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3.2 Pedagogical Agents’ Intervention 

In the context of educational games, using pedagogical agent all forms of adaptive intervention provided by 
Pedagogical agent is summarized as either cognitive or motivational (Ofut and Shabalina 2013b). 

Cognitive interventions. Pedagogical agents could posses some cognitive intervention functionalities to 
strive to enhance cognitive abilities and support the learner based on his/her user model. A number of 
subtypes of this type of intervention are distinguished below: 

- competence activation interventions, when a learner is stuck in a certain task because results led to 
the assumption that the learner possesses the necessary Skills, the temporary inactive skills could be 
reactivated. E.g. the agent can hint the learner “we have come across this issue already before”; 

- competence acquisition interventions, when the agent concludes that the learner lacks certain skills, 
the agent could provide the necessary information; 

- problem solving support, this intervention consists of providing support in an ongoing problem 
solving process via hints and indications that bring the learner closer to the solution; 

- progress feedback. In this intervention the agent provides the learner with information about 
learning progress of the game (through NPC or scoring mechanisms). This fosters monitoring and 
reflection on the learner’s own performance; 

Motivational Interventions. Pedagogical agents could also possess some motivational interventions 
striving to enhance and retain the learner’s motivation and engagement on a high level. A number of subtypes 
of this type of intervention are distinguished below: 

- praising interventions consists of congratulating the learner after successfully completing a game 
level, the learner is awarded a certain scores; 

- encouraging interventions could be applied in case of failure. To promote further attempts to find 
the solution, the learner is encouraged to try again by the agent; 

- incitation intervention is to foster motivation by announcing pleasing outcomes as rewards; 

- affective interventions foster emotional-affective aspects of the game and social interaction with 
other game characters; 

- attention-catchers, if the agent detects a decreasing attention through interpretation of the learner’s 
actions, it could introduce unexpected changes or incidents to increase variability and keep the game 
interesting. 


4. USING PEDAGOGICAL AGENT TO INFER LEANER 4 S MOTIVATION 
AND EMOTIONAL STATE 

4.1 Pedagogical Agent as Motivational-Oriented System 

Motivating students is a major issue for current ITS (Qu 2005). Modern theories of motivation such as Self 
Deterministic Theories (Reeve 2004) have shown the positive motivational impact of autonomy-support. 
Autonomy-support as a way of enhancing motivation is one of the major points of the Self Determination 
Theory (Reeve 2004) among others. It is on that note that we support the introduction of pedagogical agents 
in educational games to create motivations by acting as an autonomy-support tool for learning. One of the 
methods used by agents to provide autonomy is by encouraging the learner to make choices during game play 
activity. Alternatively, any action which controls or restrains the learner’s behaviors has a negative impact on 
him/her sense of autonomy, which results in lowering his/her motivation to persist in the activity. 

“ Students with autonomy-supportive teachers compared to student with relatively controlling teachers, 
show greater mastery motivation, perceived competence and intrinsic motivation, greater conceptual 
understanding, higher academic performance, and greater persistence in school ” (Reeve 2004). 
Pedagogical agent in educational games with increasing teaching autonomy support results in an 
enhancement of Learners’ engagement in a learning task. 
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4.2 Pedagogical Agent as an Emotional-Oriented System 

Emotion is an important aspect of human intelligence that should be considered while designing pedagogical 
agents in educational games, because it plays a very important role in the human teaching and learning 
process. According to Scherer (Scherer 2000), emotion is the relatively brief episode on synchronized 
responses for most or all organic systems for the evaluation of an external or internal event as being of major 
significance. 

Pedagogical agents with emotional intelligence possess abilities such as: recognizing the current 
emotional state of the learner and addressing it, provide intelligent interactive learning environment, and can 
also create a more personalized and friendly environment for learning. 

In our work (Ofut and Shabalina 2013b) we have strongly argued that the inclusion of emotional features 
in pedagogical agents is with high importance for improving the level of interaction in man machine 
communication system. 


5. IMPROVING PEDAGOGICAL AGENT INTERACTIVITY: FROM 
PEDAGOGICAL AGENT TECHNIQUES TO TUTOR AND TUTEE 
AGENTS’ TECHNIQUES 

We have decided to improve pedagogical agent interactivity techniques by introducing two types of 
pedagogical agents: Tutor and Tutee Agents to create a high social and collaborative atmosphere in a DEG 
environment. 

The Tutor Agent (TrA) primary function is to interact directly with a learner and explicitly guide him/her 
through the domain. These pedagogical agents are applicable in teaching components and user interfaces. 
They encourage the learner by providing feedback within the learning environment. 

The Tutee Agent (TeA) is an agent that serves as a peer-mediated learning agent. This type of agent is 
positioned in the user interface to act as interactive partners in a learning process. One of the major objectives 
of this agent is to provide collaboration with each other in a game play environment. 

We are of the opinion that the absence of learner’s companion agent and lack of social interaction in the 
games give rooms for low motivation and poor emotions. That is why we have decided to diversify the 
functionality of a Pedagogical agent by introducing a Sub-agent (TeA) as a learner’s companion (Ofut and 
Shabalina 2013a). 


6. IMPLEMENTATION OF AGENT-BASED TECHNOLOGY IN 
ADAPTIVE EDUCATIONAL GAME 

In this session we discuss the implementation process of Tutor and Tutee agents in an Adaptive Educational 
Game. The Goal of the Game is to make the learning of programming as simple as possible by not just using 
a pedagogical agent as an instructor, but also by introducing a learner’s companion agent to serve as a peer to 
the learner to improve interactivity. 

Responsibilities/characteristics of TrA 

1 . Introducing the learning task at the beginning of the a game level 

2. Providing coaching instructions to learner by communicating verbally and none verbally. 

3. Providing assistance to learner when the learner is slow to act or repeatedly unable to get a task 
right. 

4. The TrA would motivate the learner by appraisal, when he/she is doing well. 

5. The TrA would express emotions when the learner fails a particular task by calling the learner by 
name and encouraging him/her to try again. 

6. TrA provides progress feedback intervention to the learner with information about learning progress 
of the game (through NPC or scoring mechanisms). This fosters monitoring and reflection on the 
learner’s own performance. 
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Figure 1 . DEG interactivity with Tutor and Tutee Agents 

Responsibilities/ characteristics of TeA 

1. TeA provides problem solving support in an ongoing problem solving process via hints and 
indications that bring the learner closer to the solution. 

2. TeA Provides solution to a task which the learner could not complete within a reasonable number of 
actions/time. These ensure the game flow continuity. 

3. If the learner is presented with an unclear or ambiguous task the TeA can be contacted for clarity. 

4. The TeA can attempt to provoke the learner’s reflections about his/her own abilities, thinking 
process and confidence 

5. The TeA can also congratulate his co-learner in the case of success to motivate him/her 

6. The TeA could encourage the learner in case of failure. To promote further attempt to find solution. 

The Game’s Interface: The Java Pyramid Game is a one player game and the task of the player is to 
demolish the pyramid completely at every game level. 
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Figure 2. Java Pyramid game interface showing Tutor and Tutee Agents interactive approaches. 

In the first instance the player is welcomed by the Tutor Agent and required to enter his/her name. This 
would enable both the Tutor and Tutee Agents to refer to the player by his/her name. 

The Tutor Agent can now call the player by his/her name and introduce the Tutee Agent to the player. 

The interactivity level of the game at all level is high as both the TrA and TeA communicate with the 
learner based on his/her action during game play. The interaction is modeled in such a way that the emotion 
and motivation is boosted at every point of intervention by both agents. The social interaction between the 
agent and learner is either in verbal (Voice) or non verbal (dialog box) form. 

The TeA knowledge is modeled to have a better understanding of every task in all the game level in order 
to mentor the learner as well as provide collaboration support. The TrA interacts directly by coaching the 
learner throughout the game play. 


7. CONCLUSION AND FUTURE WORKS 

This paper presents the relevance of Pedagogical agent’s technology as an approach in enhancing the 
interactive learning in a game based environment. We have compromised a lot in the pedagogical agent 
strategies and we stick to two, which are agent collaboration strategy and learner’s companion/peer agent. 
Both the collaboration strategy and learner’s companion are being influenced by the Tutee Agent (TeA) in 
the game environment. The idea is that we are taking advantage of the social interactions between the learner 
and the TeA to improve the game interactivity. Currently we are designing a game prototype to demonstrate 
the effectiveness of our approach. 
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ABSTRACT 

Due to technological development and intensification of integration processes all over the world, people from different 
cultural backgrounds have more opportunities to maintain academic and professional cooperation. To make this 
cooperation more effective, it is important to take into consideration diverse ethnic values and their influence on 
interaction. That is why many researchers focus on the issues that arise when there is a need to use modern web 
technologies in a multinational context. This paper addresses the problem of providing education to Chinese students in a 
foreign country by means of wikis. 


KEYWORDS 

Wiki; culturally sensitive; collaborative learning. 


1. INTRODUCTION 

Web 2.0 has become an integral part of our daily lives and a focus of interest of numerous researchers but it 
seems that it still has an undiscovered pool of opportunities in the educational sphere. Numerous researchers 
discuss issues arising in connection with wikis in education; some of them touch upon cultural challenges 
posed by Web 2.0-based instruction; but few offer culturally sensitive educational models with wikis 
involved. 

The present paper describes an experiment designed to prove that wikis can be an effective instrument for 
developing Chinese students' media literacy by means of creating a culturally sensitive learning environment 
for them. 

The aim of the educational experiment was to improve media skills of Chinese students who major in 
English Literature and Philology which was achieved through the following steps: 1) to draw up a "cultural 
portrait" of Chinese students and determine the effects of the cultural features on students' performance; 2) to 
define the structure and content of media literacy and principles of its formation as a part of the professional 
competence of a philologist; 3) to work out the selection criteria for mass media texts as well as the 
principles of organizing instructional materials with Chinese students' cultural characteristics in mind; 4) to 
design exercises for the culturally sensitive learning environment and check their effectiveness. 

It was hypothesized that teaching media literacy to Chinese students with wikis involved could be 
effective if 1) it is based on the analysis of cultural characteristics of students; 2) the formation of media 
literacy includes the following components: a) media critical skills, b) media communicative skills, c) 
technological skills, d) media interactive skills; 3) it is based on the principles of developing media literacy as 
a part of the professional competence of a philologist. 

The following research methods were used: 1) analysis of academic literature; 2) analysis of regulatory 
documents; 3) analysis of course books for developing media skills; 4) observations of Chinese students' 
learning process; 5) questionnaires and tests; 6) quantified analysis of data; 7) experimentation. 
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2. CHINESE CULTURAL CHARACTERISTICS 

The cultural portrait of Chinese students was formed on the basis of the previous research in this sphere and 
our experience of teaching Chinese students (Flowerdew, 1998; Lixin, 2006; Zhao and McDougall, 2008; 
Jiang, 2008; Zhu et al., 2009; Chou and Chen, 2010; Parrish and Linder- VanBerschot, 2010; Earl and Cong, 
2011; Qian and Tian, 2013; Wang, 2013). 

1. Students' attitude to teachers. The image of a strict but fair teacher dominates the Chinese classroom. 
Chinese students are supposed to show enormous respect to teachers and rely heavily on them as a source of 
knowledge, guidance, and self-esteem. It means that the same image of power and care should dominate the 
online learning environment where Chinese students need clear instructions, teacher's assistance, and 
reasonable and immediate feedback. 

2. Students' attitude to each other. Chinese students' relationship within the classroom can be described as 
harmonious, cooperative, conflict-free. They care not only about saving their own face but also about the 
reputation of their group mates avoiding criticizing each other or showing off too much. It means that an 
online discussion between them may never start and wiki pages meant for peer reviewing will stay blank or 
contain a few positive remarks. 

3. Students' learning style. The learning style of Chinese students may be characterized as needs for 
precedent, visualization, and time. That is to say, they succeed in completing a task when they have clear 
instructions and patterns to follow, acquire knowledge better with the help of visual cues, and have long term 
orientation, which means they prefer to get deeper knowledge and understanding of the subject without tight 
time constraints. 

4. Students' attitude to new technologies in education. Although Chinese students are always well- 
informed about the latest news in high-tech world, and they are good at using devices and applications, one 
should not expect that they will not have any problems when it comes to mastering some Internet tool. 
Moreover, many Chinese students are wary of online studies because they tend to consider distance education 
or any elements of it to be second-rate. That is why the implementation of any devices, applications or 
Internet tools would be highly appreciated by Chinese students if a teacher explains all the goals and real 
benefits of this procedure. 

Overall, the aforementioned features form a type of learner behaviour that can be described as the 
combination of industriousness and lack of creative learner autonomy, empathy and oversensitivity to 
criticism, ability to co-operate and fear of failure, interest to new technologies and concern about the quality 
of education these technologies can provide, accompanied by cogitative needs for clear instructions, 
visualization, and time. These characteristics should be taken into consideration as they may pose challenges 
and open prospects. 


3. MEDIA LITERACY 

At the Faculty of Philology of our University, Chinese students majoring in English Literature and Philology 
are offered a course in reading, watching, and discussing news in English. Thus, teaching this course implies 
development of students' media literacy within their professional philological competence. While doing the 
course, they find it quite difficult and demotivating which can be attributed to the design of the course itself, 
the lack of English language proficiency, the lack of media literacy (which is explained by the fact that media 
education is still at its initial stage in China), and the influence of the cultural peculiarities described above 
(Liao, 2008; Wan and Gut, 2008; Tan, 2012; Xu, 2013). The list of the problems Chinese students face when 
reading analytical reviews and features or watching talk shows and discussing them includes: 1) inability to 
spot key points of the text they read or watch; 2) learning texts by heart instead of summarizing the 
information; 4) inability to give their own opinions and develop argumentation; 3) lack of knowledge about 
cultural realias of English-speaking countries; 4) lack of active and passive vocabulary; 5) lack of contextual 
deduction; 6) lack of understanding complex grammar structures; 7) low (mostly extrinsic) motivation. 

The challenges mentioned above led to the idea of creating a model for developing Chinese students' 
media literacy with the help of a culturally sensitive wiki space in order to increase their motivation, improve 
their performance, and foster their learner autonomy, creativity, and activeness not only in the classroom but 
also in the online environment. 
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According to the model of developing media literacy within their professional philological competence, 
Chinese students have the following skills to acquire: 

1) media critical skills (analytical skills enabling students to find key content elements of mass media 
texts, such as topic, problem, facts, main idea in analytical articles and episodes of talk shows; and elements 
of users' comments, such as position, examples, and informative value); 

2) media communicative skills (communicative skills including reading and listening to mass media texts 
in English, expressing and discussing opinions about mass media texts in English, both orally and in the 
written form); 

3) technological skills contains (technological skills including collaborative doing of tasks, editing of wiki 
pages, creating wiki pages, participating in wiki discussions, communicating with other students and the 
teacher); 

4) media interactive skills (interactive skills implying organizing a simulation in the form of a TV talk 
show where participants discuss socially important issues); 

The development of media literacy shall be based on the principles of the integration of media education 
and professional education. Thus, we followed the principles of the integration of media education and 
English language learning: 1) the principle of complementary development of media literacy and 
communicative competence in English; 2) the principle of considering cultural characteristics; 3) the 
principle of critical thinking development; 4) the principle of developing skills necessary for the usage of 
mass communication tools; 5) the principle of the practical usage of skills (Masterman and Mariet, 1994; 
Sefton-Green, 2000; Bean, 2001; Hartley, 2002; Buckingham, 2003; Branston, 2006). 

The principle of complementary development of media literacy and communicative competence in 
English implies the priority of the communicative component within the media literacy of Chinese students 
and the usage of English mass media texts for the development of their communicative skills in English. 

The principle of considering cultural characteristics presupposes that students might demonstrate certain 
strengths and weaknesses when trying to acquire media literacy skills due to their cultural background. Thus, 
the educational process must be organized in such a way that turns off any challenges caused by cultural 
characteristics of students. 

The principle of critical thinking development means that students must be taught to analyse important 
social problems presented in mass media texts and formulate and maintain their own opinions. 

The principle of developing skills necessary for the usage of mass communication tools states that every 
media literate person should be able to use diverse mass communication tools for storing, processing, 
producing, and exchanging information including mass media texts. 

The principle of the practical usage of skills means that all those media critical, media communicative and 
technological skills that students acquire must be used in practice, i.e. for organizing projects, case studies, 
simulations and other types of interactive activities that imitate diverse kinds of professional communication. 

The application of the principles mentioned above enables a teacher to make the most of mass media texts 
in English chosen for developing media literacy skills, however, the classroom time is not enough for 
improving all the media literacy skills. Hence, there is a necessity of organizing students' extracurricular 
time with the help of a mass communication tool that could be also used as a means of developing students' 
technological skills and would enable a teacher to create culturally sensitive tasks. 


4. WIKIS 

As well as other similar Web 2.0 tools, wikis are perfectly fit for collaborative learning rooted in the theory 
of constructivism (Paloff and Pratt, 2005; Parker and Chao, 2007; Sandifier, 2011). However, to create a 
culturally sensitive online learning environment it is important to take into consideration all the four 
parameters characterizing Chinese students that were described above; 1) students' attitude to teachers; 2) 
students' attitude to each other; 3) students' learning style; 4) students' attitude to new technologies in 
education. 

Firstly, Chinese students need to feel the teacher's presence in the wiki: in the same way as a teacher 
dominates the classroom in China, he or she should dominate the wiki providing clear and consistent 
guidelines, providing timely consultations through wiki mail or other instruments available in the wiki, 
checking, correcting or commenting on homework, setting deadlines and giving marks. 
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Secondly, wikis are an ideal place for a harmonious, cooperative, and conflict-free learning that Chinese 
students strive for. It is easier to save face online because students have time to think, and wikis provide an 
opportunity to work a) in groups or b) individually but in collaboration ( which means each student does some 
part of a common task and bears responsibility for his or her part individually): when students share a task, 
they have less psychological pressure because they either bear responsibility as a group (not individually) or 
at least get their peers' assistance when working individually. 

Thirdly, wikis suit Chinese students' learning style as they meet their needs for precedent, visualization, 
and time. All these needs can be easily satisfied in the wiki space which presupposes the use of visual content 
and asynchronous communication, which make Chinese students feel more comfortable than in face-to-face 
communication in the traditional classroom when they have to rely more on verbal perception and produce 
answers immediately. Both of these wiki features complemented by clear organization of learning activities 
and well-formulated tasks can make wikis a smart addition to face-to-face studies. 

Fourthly, the usage of wikis in the educational process can be motivating for students if a teacher explains 
how to use the wiki space and the main goals of using this tool. 


5. CULTURALLY SENSITIVE WIKI ACTIVITIES 

For the purpose of designing culturally sensitive wiki activities the following selection criteria for mass 
media texts were used: 1) compliance with communicative and general cultural competencies of Chinese 
students; 2) being in accordance with ethical norms; 3) diversity of themes and formats (printed media, audio, 
video, illustrations); 4) analytical texts containing not only information but also authors' opinions; 5) 
authenticity; 6) informative value. 

The principles of organization of instructional materials included the principle of thematic organization 
and the principle of considering cultural characteristics of students. 

According to the principle of thematic organization the instructional materials were grouped into nine 
units with articles on the following topics: Sport (Health, Olympics), Culture (Social Networks, Problems of 
Education), Science and Technology (Achievements, Ecology), Business (Products of Big Companies, 
Employment). Each unit also contained comments on different media critical topics to develop students' 
media critical skills: Types of Mass Media Texts, Headlines, Structure of Article/Video, Key Elements of 
Article/Video (topic, problem, facts, idea). Summary of Article/Video, Key Elements of Comment (position, 
example, informative value). Opinion about Article/Video, Participation in Discussion. 

According to the principle of considering cultural characteristics of students, tasks were divided into two 
groups (wiki activities and in-class activities) with Chinese students' learning style in mind. For example, 
listening activities were presented as wiki tasks for homework because Chinese students are used to rely 
heavily on visual materials in their educational system and, as a result, their listening skills need considerable 
improvement, so it is much easier for them to save face when they do listening tasks at home in the wiki 
where every student may watch a video in the task as many times as necessary. 

Using the principles mentioned above, we have designed a set of activities for wiki and in-class studies. 
While learning activities in the wiki space were considered to be the preliminary and closing stages for in- 
class studies including vocabulary work, information search (realias), revision of media critical theoretical 
comments, listening and writing practice, acquisition of technological skills, in the classroom students 
mainly learnt theory, studied articles and did normal pre-reading, reading, and post-reading activities, which 
also included lexical and grammar exercises as well as speaking activities (reports and discussions). 

In order to develop media literacy, the wiki site named Wikispaces was chosen 
(http://www.wikispaces.com) and used to create a wiki space. The wiki space had the following structure: 1) 
home page (contained teacher's announcements about new home tasks posted in the wiki space); 2) thematic 
pages (contained tasks for collaborative learning); 3) checklist (contained an assessment chart with students' 
marks). 

As soon as students got a notification via wiki mail, they opened the home page of their wiki space and 
went via the most recent link to a thematic page. The theme of a page depended on the issues discussed in 
articles chosen for an upcoming class. Each thematic page contained links for downloading handouts for 
upcoming classes and the set of activities for individual work and work in mini-groups depending on the 
difficulty of an activity. The wiki and in-class activities are described below: 
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1. Wiki activities: 

1) Illustration (warm-up activity for mini-groups, done before an upcoming class). Students were asked to 
think in mini-groups of a picture posted on a thematic page, and explain its irony as well as use it to predict 
key topics and problems touched upon in mass media texts chosen for in-class studies. 

2) Vocabulary and Realias (vocabulary activity for individuals, done before an upcoming class). The 
activity was done individually but in collaboration with each other. The teacher created a list of theme-based 
expressions and realias (culture-specific information) derived from articles (chosen for an upcoming class) 
and a video (posted on the same thematic page). Students were to translate expressions into Chinese and 
provide Chinese definitions for realias. The list was divided into portions and shared by students so each 
student did not have to do the whole list but a small part of it and had to post translations and definitions on 
the thematic page before a specified time in order for other students to see his or her portion of work and, 
consequently, see other students' posts. As a result, students did the vocabulary work in collaboration which 
facilitated their homework and in-class studies, and implied joint responsibility. After the specified time, the 
teacher checked the posts, did corrections, and put the marks into the assessment chart immediately, so 
students could get the immediate feedback on the same day. In the classroom, students could check the list of 
expressions and realias, when necessary, getting the access to the wiki space via smartphones and tablets. 

3) Drills (vocabulary activity for individuals, done before an upcoming class). Again the task was done 
individually. The teacher posted a number of exercises created on the basis of the vocabulary list (gap-filling, 
matching, multiple choice, odd one out, rephrasing, etc.) on a thematic page. Each student got some kind of 
exercise which included a part of the vocabulary done by other students and he/she had to post the answers 
on the page before a specified time for the teacher to check, do necessary corrections, and put a mark into the 
assessment chart. 

4) Video (listening activity for mini-groups, done before an upcoming class). Not all the students were 
equally good at listening, so they were divided into mini-groups for doing this exercise. The task was chosen 
for wiki work because in the classroom students usually did not have enough time to watch a video as many 
times as they needed. Watching and listening were facilitated by the previous vocabulary work in the wiki 
space. Students were to use a link on the thematic page to go to Youtube or any other site with video content 
and watch a short video thematically connected with all the previous tasks and the articles for in-class work. 
Afterwards, students were offered to post answers to questions about the content of the video or do a true- 
false exercise on the thematic page. The homework was checked and assessed by the teacher. 

5) Link to China (speaking activity for individuals, done before an upcoming class). Students were 
offered a few questions in connection with the theme of the page and the upcoming class. The main idea of 
the task was to make students think of the issues in question in connection with their own country: if a 
situation which takes place in some English-speaking country is the same in China or not, and why. So, 
students had enough time to think and look for the relevant information, if necessary, before discussing it in 
the classroom. 

6) Theory Revision and Writing (writing activity for mini-groups, done after a class). In the classroom, 
students got media critical theoretical knowledge and practical skills but in the wiki space they were to do the 
revision of the material. Depending on the material students studied in the classroom, they were asked either 
to formulate a problem using prompts, match problems and facts, read a given comment for a mass media 
text and define the commenter's position, write one’s own comment in response to a given comment and so 
on. The homework was checked and assessed by the teacher. 

7) Wiki skills (set of instructions for students to learn how to use wiki instruments). 

2. In-class activities: 

1) Illustration (warm-up activity for mini-groups, done at the beginning of a class). This exercise was a 
counterpart of the first activity with a picture in the wiki. It was used to check how students interpreted the 
picture at home in mini-groups and used it to predict the topic and problems of an upcoming class. 

2) Check Vocabulary and Realias (done by the whole group). The exercise was used to check meanings 
of new theme-based words and definitions for realias that students wrote in the wiki. 

3) Theory (done by the whole group). Blocks of media critical theoretical comments were included into 
the set of in-class activities for students, firstly, to learn about the key elements of mass media texts for 
improving their analytical, reading, and watching skills; secondly, to learn about the structure of opinions that 
they need to create on the basis of mass media texts and ways to develop argumentation in discussions for 
improving their speaking and writing skills. 
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4) Group Speed Task (done in mini-groups). This task included pre-reading and reading activities. The 
pre-reading activity helped students to predict what the text was going to be about. As for the reading 
activity, it was done in mini-groups who had one article cut into several pieces (for example, topic sentences 
and bodies of paragraphs) that needed to be matched and put together in the right order as fast as possible. 
The task was aimed at developing students' media critical and reading skills. Its content depended on the 
theory they had learnt. After completing the task, students did exercises with evaluative words as well as 
grammar exercises. 

5) Individual Task (done individually). The task included the same pre-reading activities as in the 
previous task but students had the whole text of the article which they were to read and analyze individually. 
After reading the article they had to do exercises based on the theory they had learnt. 

6) Link to China (done by the whole group). This post-reading task was a counterpart of wiki activity N°5. 
Before each class students were offered a few questions connected with the theme of the class. They had to 
think of these issues at home to be able to compare situation described in articles with similar situations in 
China. Thus, students used their ideas and participated in discussions about the articles they had read 
improving their speaking skills. 

At the end of the course, students participated in a creative activity. They were offered to participate in a 
simulation reproducing a situation of a real talk show using all those media skills that they had acquired. 
They presented the simulation in the classroom but all the preparatory job had been done at home with the 
help of the wiki space which was used to post the scenario of the talk show and words for the roles so the 
teacher could check them and post tips and recommendations. 

All in all, the organization of the educational process was in line with Chinese students' cultural 
characteristics. Firstly, while working autonomously at home, students had clear instructions to follow (in 
terms of using the wiki space and doing the activities), they could communicate with the teacher if they had 
any problems and could see that their homework was constantly and immediately checked by the teacher. 

Secondly, the wiki environment prevented students from losing their face as they had enough time to 
refine their posts and worked either in mini-groups or individually but collaboratively, so, they could feel 
their group mates' support all the time. 

Thirdly, students' cogitative needs for precedent, visualization, and time were satisfied as they got clear 
instructions for using the wiki space and doing activities, used visual content, and communicated 
asynchronously, so they had enough time to work on tasks and think of their answers. 

Fourthly, there were a couple of introductory classes for students where they learnt how to use the wiki 
space (registration, creating and editing pages, uploading/downloading files, posting comments, using wiki 
mail) and where they were told about the main goals of using this tool (well-organized and user-friendly 
storage of study materials where they can be easily found anytime and anywhere, no need to print study 
materials which can be easily read in a smartphone/tablet; collaborative language learning activities which 
enable them to share knowledge and, thus, spend less time on homework, charts with marks which enable 
them to follow their progress). 


6. EXPERIMENT AND RESULTS 

The set of activities described above was tested in the spring semester of 2013. There were two groups of 
Chinese students in their third year of University who participated in the experiment (18 students in the 
control group, and 19 students in the experimental one). Students' proficiency in English in both groups was 
B1-B2. The experimental group studied offline and online using the above mentioned set of activities while 
the control group studied offline using the textbook "Reading the News" (Sharma, 2007). 

The experiment included a pre-experimental test, experimental trainig, and a post-experimental test. The 
experimental training took 13 face-to-face classes, 10 wiki assignments, and one final face-to-face class. 

The aim of the pre-experimental test was to measure students' media literacy level before the 
experimental training. The test included the following tasks: a) read an article, provide a summary, and give 
one's opinion about the problem described in it; b) watch a video, answer the questions, write a short 
summary and one's opinion about the problem discussed in it; c) register in the wiki 
(http://www.wikispaces.com/) and create one’s profile; d) make a dialogue on one of the topics (Sport, 
Culture, Science and Technology, Business). 
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The following parameters were used to define the levels of media literacy skills. For each parameter a 
student got one point. 

1. Media critical skills: a) knowledge of types of mass media texts; b) knowledge of lexical and 
grammatical features of news headlines; c) knowledge of structure of mass media texts and their key content 
elements (topic, problem, facts, idea); d) ability to tell what a mass media text is about using headlines, first 
and last paragraphs; e) ability to define the main topic of a mass media text; f) ability to define the main 
problem of a mass media text; g) ability to find and connect the facts that prove the problem exists; h) ability 
to define the main idea of a mass media text; i) ability to read an Internet comment on a mass media text and 
define the user's position regarding the problem described by the author of the mass media text; j) ability to 
find those examples that a user provided to prove his/her position in a comment; k) ability to define how a 
user feels about the informative value of the mass media text he/she commented upon. 

2. Media communicative skills: a) knowledge of thematic vocabulary; b) knowledge of evaluative words 
and expressions; c) knowledge of cultural realias; d) knowledge of discourse markers; e) knowledge of 
grammar rules; f) ability to distinguish between facts and opinions while reading/listening to a mass media 
text; g) ability to make a short summary of a mass media text (orally, in the written form) defining its topic, 
problem, facts, main idea; h) ability to give one’s opinion about a mass media text including one's position 
concerning the problem of a mass media text, one's examples, and description of the informative value of this 
text; i) ability to make a dialogue discussing socially important issues; k) ability to use discourse markers in 
one's monologues and dialogues. 

3. Technological skills: a) knowledge about application of wikis; b) knowledge about wiki functions 
(creating and editing pages, uploading and downloading documents, checking recent changes, posting 
comments, sending wiki mail etc.); c) ability to register and create one's profile; d) knowledge of wiki space 
structure and ability to look for study materials; e) ability to download and upload materials in a wiki space; 
f) ability to check recent changes and restore older versions of documents; g) ability to post comments; h) 
ability to use wiki mail to keep in touch with other students and a teacher. 

4. Media interactive skills: a) knowledge of key stages of organizing a simulation; b) knowledge of a 
simulation scenario; c) knowledge of discourse markers used in a scenario; d) ability to choose a theme for a 
simulation and find relevant mass media texts; e) ability to create one's utterances for a chosen role; f) ability 
to change a scenario in a creative way without destroying its main structure; g) ability to present a simulation; 
h) ability to reflect on simulation experience. 

The results of the pre-experimental test are presented below: 
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Figure 1. The levels of media literacy and media literacy skills in the control and experimental groups (%) 

The analysis of the first diagram that presents the levels of media literacy shows that before the 
experiment these levels had been low (23 and 25,4%) with not much difference between the control and 
experimental groups. The second diagram presents similar levels of media critical, media communicative, 
and media interactive skills in both groups which vary from 25 to 35% while technological skills are the least 
developed (about 8%) which is easily explained by the fact that students had not used wiki sites prior to the 
experiment. 

The aim of the post-experimental test was to measure students' media literacy level after the experimental 
training. The test included the following tasks: a) read an article, provide a summary, and give one's opinion 
about the problem described in it; b) watch a video, answer the questions, write a short summary and one's 
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opinion about the problem discussed in it; c) answer questions about wikis (for the experimental group only); 
d) make a dialogue on one of the topics (Sport, Culture, Science and Technology, Business). 

The same parameters were used to define the levels of media literacy skills. The results of the post- 
experimental test are presented below: 



■ Control group 

□ Experimental 
group 



Merit - media critical skills, Mcom - media communicative 
skills. Tech - technological skills, Minteract - media 
interactive skill 


Figure 2. The levels of media literacy and media literacy skills in the control and experimental groups (%) 

The first diagram proves the positive dynamics in both groups: the level of the control group rose by 10% 
while the level of the experimental group increased considerably, by 50%. The second diagram demonstrates 
the highest levels of media interactive skills in both groups (53% in the control group, 80,25% in the 
experimental group). The levels of technological skills differ significantly (about 10% in the control group, 
76% in the experimental group) because the wiki site was not used in the control group. As for media critical, 
media communicative, and media interactive skills, the experimental group was able to improve their 
performance by 40-55% while the control group managed to increase their score by 10-20% only. 

The results of the experiment showed that the wiki space (along with the changes of the course content) 
contributed to the development of Chinese students' media literacy as a part of their philological competence. 
In other words, students from the experimental group gained better media critical, media communicative, 
technological and media interactive skills than those in the control group who studied offline only. 

After the experiment, students were given questionnaires for getting their feedback concerning the usage 
of the wiki space. Chinese students made a note of the following advantages of implementing wikis into the 
traditional educational process: 1) more convenient than e-mail because the wiki space is well-organized and 
it is easy to find links to study materials; 2) even if students miss a class, they may find all the handouts in the 
wiki space; 3) no need to print study materials; 4) the wiki space is convenient for multi-tasking (doing 
homework, sharing homework, being in connection with the teacher and group mates via wiki mail, check 
one's study progress); 5) convenient for team work (sharing homework, feeling of belonging to a group, one 
does not feel lonesome when doing homework); 6) easy to follow one's study progress and compare one' s 
marks to those of other students; 7) opportunity to edit pages, change something oneself. 

As for the wiki drawbacks, students mentioned 1) bugs; 2) lack of interesting applications integrated with 
the wiki space; 3) irresponsibility of some students who did not do their portion of homework in time and let 
other students down; 4) discomfort caused by the fact that other students could see one's marks in the 
assessment chart; 5) plain design. 


7. CONCLUSION 

Overall, the positive results of the experiment as well as generally positive feedback from students showed 
that a culturally sensitive wiki space could work if the following points are taken into account: Teacher 
Support (clear instructions, assistance, feedback). Peer Support (sharing homework, work in mini-groups, 
face-saving online environment). Focus on Cogitative Needs (clear instructions and patterns, visual content, 
flexible time constraints). Technology Insight (introductory classes for students to learn about wiki 
application purposes, wiki functions, and wiki activities). 
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It is also important to note that wikis and the aforementioned principles of designing a culturally sensitive 
wiki space may not work for students of all ethnic backgrounds and may not be fit for all educational 
purposes. 

Anyway, the present case demonstrates that it is worth a try if students and teachers face certain 
challenges (including those connected with cultural differences), and traditional classroom techniques and 
tools do not provide sufficient help. 
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ABSTRACT 

This paper reports on a study designed to examine the dialogic processes teachers used to sustain focused discussions, 
using questioning techniques and Interactive Whiteboards (IWBs). IWBs and their related technologies such as plasma 
touch screens and projected tablets have passed through several phases of implementation as classroom objects, from 
initial adoption by a few enthusiasts, through the building acceptance phases, and finally a phase of maturity 
characterized by widespread use. This study shows that teachers who value shared collaborative learning spaces, can 
draw on distributed cognition, and promote sustained thinking, cognitive engagement and that this enables development 
and refinement of students insights. 


KEYWORDS 

Interactive Whiteboards, Shared Cognition, Dialogical teaching moments, 


1. INTRODUCTION 

When IWBs were first introduced, the early studies in the United Kingdom focused on pedagogies related to 
interactivity that involved teacher and students, and between students (Smith et al, 2006; Tanner et al, 2005; 
Moss et al, 2007). Teachers tried to realize the affordances of IWBs. Criticisms focused on IWBs easily 
being absorbed into the existing practices of teachers without pedagogical changes, and reflected past 
practices of teachers. A major UK study concluded “IWBs are mainly being used: as a data projector which 
can navigate to multiple screens; as a surface which can generate a dynamic rather than static form of 
display; to enhance presentation from the front of the class” (Moss et al, 2007 p. 5). IWBs may have lead to 
less thinking time when the teacher retains control. This study sought to see if more thinking time could 
happen. 

Studies have tried to evaluate the IWBs impact on pedagogy, on uses to reinforce whole class teaching, 
and teacher engagement with the “surface features” of interactive teaching (Rudd, 2007; Smith, Hardman & 
Higgins, 2006); Mercer, Hennessy & Warwick, 2010). Others are equivocal, while claiming some positives. 
Warwick and Kershner, (2008), commented: “Research evidence on educational gains from IWB use across 
both primary and secondary phases of schooling is mixed” (p.269). 

There has been limited understanding about how pedagogy develops with IWBs in the UK, USA and 
Australia (Hennessy et al 2007). In Australia attempts to classify IWB impacts in terms of a hierarchy of 
pedagogical skills, led to questions of whether the affordances of the technologies could stimulate higher 
order thinking skills through interactivity (Vincent & Jones, 2008; Sweeney, 2008)), similar to other studies 
quoted here, these sought to examine the impact of the technology in changing pedagogy. 

This paper presents one teacher’s use of the IWB to support shared cognition. Perhaps earlier 
disenchantment with IWBs arose because the wrong question was being asked: what can the technology do 
for teachers/students? A more useful question might be: what can the technology do with me/students? This 
teacher’s lesson review indicates that she works with the affordances of the technology, to support 
dialogically meaning making moments in her classroom. Her pedagogical work aligns with the seven 
pedagogical practices for using technologies (Osborne and Hennessy, 2003). This teacher’s in class voice, 
and again her voice in the interviews, indicates she understood that a shared language needed to be created 
before students could engage in discussions, and before they could question and think about what was 
happening (Redman, 2010). Her class can be seen to be working with ideas afforded by the teacher using the 
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IWB, because they have the necessary language needed to enter collaborative and challenging thinking 
spaces. 

Luckin (2008) and Beauchamp (2011) used the same concept of ‘working in partnership’ with machines. 
This is done with a view to redefine and enhance performance as noted by Salomon, Perkins and Globerson, 
(1991), now this includes IWBs and digital interactive technologies. Luckin and Beauchamp also take 
account of the local setting and culture within which tools are embedded. Luckin (2008, p 451), describes 
learning taking place within an ecology of resources, ‘a set of inter -related resource elements, including 
people and objects, the interactions between which provide a particular context’. This message, with a few 
exceptions, became lost in the euphoria of the early adoption of IWBs. It is now being re-examined. 

A possible pedagogical contribution that IWBs can make to pedagogy practices is by enabling students to 
work through their ideas both verbally, and, graphically. Warwick and Kershner (2008) report on a teacher 
who linked the IWB’s multimodality affordances with the IWB, using it as “supportive of the collective 
memory of the group’’ (p. 276). The thinking of the user is made visible to all, empowering students to more 
effectively, and knowingly, participate in collaborative reasoning and hypothesis testing that may well “go 
well beyond those afforded by more established classroom devices.... [the IWB] provides a dynamic and 
manipulable object of joint reference which offers new forms of supports for ‘intersubjectivity’” (Hennessy 
et al, 2007 p.284). 

Hennessy has shown IWBs assist teachers development of a culture of shared dialogue leading that can 
lead to ‘shared cognition’ in classrooms (Hennessy et al, 2014). Notably, Hennessy is reporting on a study 
that was conducted with teachers who already had belief and practices revolving around student dialogue. 
The project concluded that when teachers purposefully create the right conditions to support risk taking and 
changing of minds, rich new forms of dialogue and activity emerge, and both at the board and away from it. 
Hennessy states the pre-existing teachers’ views about learning, and the classroom cultures they nurture, are 
central to developing the more productive dialogic uses for IWBs. Their willingness to at least partially 
relinquish control over the IWB and to take the time to consider students’ views was identified as critical. 
This suggests that a dialogic culture has to exist first. 

Classroom situations in which collective intelligence and shared cognition can potentially be fostered 
through IWB use is reported on by Alvarez et al (2013). Harnessing group intelligence to extend students’ 
productivity and depths of understanding, and of distributed cognition by sharing thinking and insights 
appears to be possible only once the teacher creates a shared platform. These are affordances offered by the 
IWB, and other devices. These activities were rarely seen, or documented, in the early phases of IWBs use, 
noted by most of the authors already mentioned. 

Alvarez et al (2013) observed Swedish teachers, in upper primary schools, use a framework that 
supported collaborative and shared solutions. Using the IWB, teachers provided students opportunities to 
share and deepen their thinking. These students reported that they “had a better understanding of how to 
solve a problem as a result of the greater communication they had with their peers and their teacher” (p. 376). 


2. METHODOLOGY 

2.1 Project Outline, Participants and Data Collection 

This project originated as part of ongoing research within the Melbourne Graduate School of Education on 
the quality of teacher-student interactions and teacher-teacher with technologies generally (Chandler & 
Redman, 2013; Delaney, Trapani, Chandler & Redman, 2014). It sought to examine the dialogic processes in 
three classrooms in a primary school when confident teacher IWB users incorporate technologies. In depth 
reporting on a classroom teacher and her interpretations of her actions are reported on and examined here. 
Drawing on the work of Hennessy et al (2014) and Mercer, Hennessy & Warwick (2010) the project focused 
on how teachers developed skills mediated by the IWB to support dialogic interactivity. It examined how 
they built a dialogic pedagogy that actively develops and extends learners’ contributions, so that they jointly 
construct knowledge. The aim of this research was to observe pedagogical responses to the affordances of the 
IWBs, especially in relation to teacher questioning and teacher organisation of the thinking dialogues. The 
three teachers involved in one primary school were observed but not part of any training program. The 
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purpose was to examine the impact, if any, of the IWB affordances on these teachers’ dialogues with their 
students, and any evidence of the IWB contributing to shared cognition across groups of students. 

IWBs have features that are potentially different from a standard classroom display and knowledge 
sharing facilities. The ‘potentially’ needs to be there because there are many studies that point out that the 
IWB can easily be used exactly like the older technologies such as screens and conventional whiteboards (eg 
Moss et al, 2007, Higgins et al, 2005). The IWB does, however, have the potential affordances of enabling 
shared thinking across groups; interactive motion; audio feedback; and animated feedback among others. 

Three teachers participated, from Prep (year 0), year 3 and year 6. They were identified and selected by 
the school as ‘accomplished’ practitioners. They were recognised for the quality of their interactions with 
students, and as regular and confident users of IWBs. In two cases the teachers had more than six years 
experience of teaching, and one teacher was a new graduate with an outstanding reputation from her two-year 
postgraduate Master of Education course. The researchers observed three lessons with each teacher, filming 
the lessons and coding all the uses of the IWBs. 

Data collection was conducted in as naturalistic manner as possible. Filming took place discretely at the 
rear of each classroom. Teachers volunteered the lessons they wished to have filmed. Teachers were not 
trained or tutored in any way, and no alterations to the timetable schedule occurred. The researchers fitted 
around the needs of the teachers. This not to ignore the fact that any intrusion, like a camera and researcher, 
into the complex system that is a classroom, is likely to have some impact on the learning environment. 

Teachers had freedom to choose the lesson topic, interestingly, the bulk of the observed lessons, were in 
mathematics. Each teacher was briefly interviewed before each lesson to establish the goals for the lesson. 
Each video film was then edited to isolate occasions on which interactive dialogue of various sorts occurred. 
A little later, but always less than 7 days, each teacher undertook a video-stimulated interview using the 
edited film, and was asked to comment on and explain the actions observed. 

The interviews were then analysed to focus on all the areas where the teachers were able to articulate that 
learning was happening due to the sharing of knowledge and where meaningful dialogue and questioning 
arose from the work with the IWB. 


3. RESULTS 

The outcomes of this project derive from the observations of, and interviews with, three expert teachers. The 
data collected was extensive and too complex to fully report here. This report concentrates on points of 
intersection between teacher questioning and the IWB, and the IWB intersections with the students, and 
evidence of shared cognition by the students. 

The teacher reported on in detail here is a graduate teacher, teaching grade four, and she has selected a 
mathematics lesson. The lesson focused on moving from angles as objects to measuring of angles of rotation. 
This teacher was in her first year of teaching, having completed a two-year post-graduate Masters program. 
In this initial teaching course she had excelled and observations in her classroom belied her inexperience. 
The mathematics lesson being used for the case study was introducing the use of the protractor and the 
concept of measurement of angles. 

Here the transcripts provide a sense of the lesson structure. The following transcript comes from excerpts 
from video recording. The teacher commences the lesson by raising a protractor. 

Teacher: Today we are going to be using this. What is a protractor used for? Tell me? Pretend you 

are explaining what the protractor is to someone who has absolutely no idea. 

Student C: Angles 

Teacher: Angles? What do you mean by angles? I understand what an angle is. So a protractor has 

something to do with angles. Let's use a full sentence to describe what it might do. (Student N) do want to 
have a go and build on (Student C’s) 

Student N: I've got two things. One, you can get it on a computer and two; it's to show the degrees of 

the angle. 

Teacher: Beautiful, so you're measuring the degrees of that angle okay? 

Now the teacher focuses on the IWB where she has an angle drawn. 

Teacher: Can someone please draw for me where the angle actually is on this diagram? 
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Teacher makes time to establish the specific ideas that she wants the children to consider and share. She 
has used the IWB as a conventional board. The angle could have been on any board, and most of the 
interaction has been verbal. At this point comes a dramatic turn in the lesson. Next the teacher brings up a 
very large moveable protractor on the IWB while Student S volunteers to come up to the board. The teacher 
now moves to one side away from the board, and only rarely returns to it for the remainder of the lesson. 

Teacher: So that is what we're actually measuring. Now I have this protractor but I have no idea how 

to use it. Has anyone used a protractor before? Ah, so we have a few experts in here. Let's see if we can share 
our knowledge. If I was to measure this angle where do I put the protractor? Who knows? 

Student M: On the bottom line and next to the point 

The teacher has focused on the IWB now showing a projected large protractor that she can move around 
the screen. She now inquires from the students and asks them: 

Teacher: What do I do? Put the two points together, but there are two sets of numbers here and I'm 

really struggling to know why two sets of numbers, and which ones are I use. 

Gesturing she now points to some of the children inviting them to. Have a guess, have a guess, I don't 

know. 

The teacher has taken up a storyline, of ‘not knowing’ and she is questioning the students, prompting their 
thinking and asking what it is she needs to do. She has used the pronoun ‘I’ signaling to the children she is 
the one who needs help (Redman & Fawns, 2010). She has positioned them as helpers, saying ‘we have a few 
experts in here’, and explicitly asked for sharing cognition from them, ‘we can share our knowledge’. 

After the lesson the teacher is reviewing the video and she is responding to questions in a conversational 
style approach. Here she is reviewing Video Clip. The teacher is pretending that she doesn't know how to use 
the protractor to measure an angle and asks the children. 

Interviewer: Just unpack that for me again what you did there to make them... 

Teacher: Just in terms of me playing dumb? 

Interviewer: That particular one why do you play dumb there? 

Teacher: Just because I feel like that it's something that they can come to themselves. They've seen 

what protractor looks like. Ideally in this situation it would be good if they could have all had their own 
protractor in their hands, and then they could see the structure of it and make the own conclusions from using 
it. Because well its not obvious but you could come to the conclusion that that's where you've got to put the 
point because it's the intersections of the lines, so it's a way for them to come to that understanding 
themselves, but also the sharing In terms of the collaboration. Those ones that are experienced with the 
protractor can share their knowledge and the others can use that time to really build up their knowledge. 

After explaining that she seeks for her students to support and assist each, now she reviews the in-class 
Video Clip recording. It shows her using IWB to investigate where to place the protractor on the surface. 

Interviewer: So you have now started to make the board support your question. What happens now? 

Teacher: They look at the common reference point on the board they have established that it needs to 

line up with the middle. . . . 

Interviewer: What is the board doing at the moment? 

Teacher: It is the common reference point. So it's acting as a collaborative effort, measuring this 

particular angle. The kids feel like they can infer that you mention the angles using the numbers and then 
from here I think with my questioning I specifically lead them to the point where they realize that they have 
to line up the zero with the line so you can measure the angle from zero, and so they're sort of going through 
that process themselves. 

This graduate teacher is explicitly stating how she is working with the board, she states it is acting as a 
collaborator, and she has the control, she leads with her questioning, she uses the board as central point of 
reference. 


4. DISCUSSION 

The teacher, while maintaining a flow of probing questions, withdraws to the side, and provokes the children 
to articulate their ideas by reference to the IWB. She more than once uses the device of pretending to know 
nothing in order to place the children at the forefront, allowing them to both refer to and to come up to IWB 
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to share their insights with the whole class. She claims that the IWB is ‘the common reference point’ which 
provokes collaborative development of understanding. 

As the lesson develops by using the animated protractor to build an understanding of why there are two 
180-degree scales, and finally measuring, the teacher repeats these processes. Sometimes she plays ignorant, 
sometimes she invites specific children to verbalize what they are seeing on the IWB and to share their 
insights with the class, and occasionally she steps in to assume control herself. 

In her Video Clip, the graduate teacher uses the very large protractor and introduces the problem of 
reading the scale into opposite directions. 

Teacher: It's quite difficult trying to teach them the protractor without having that sheer size on the 

board. 

Interviewer: Why is it important that they can see it easily? 

Teacher: Because I want them to understand that if they don't line it up with the zero, then it's very 

difficult to calculate how many degrees in between but if they lined up with the zero they can easily see how 
many degrees it is from there. So it's reinforcing the fact that they have to have it lined up from zero. 

In the Video Clip the graduate teacher asks how do we know whether we’re going to look at the less than 
90 scale or the larger than 90 scale for a specific angle. Student A answers that it depends on whether the 
angle is acute or not. 

Interviewer: So what did you do then with him? 

Teacher: The vocabulary. So it's fantastic when they can use the vocabulary. It's connecting the 

factors between the words. So that's a big step for him. In terms of the start of the year, he really couldn't 
even form a cohesive sentence. To deal with these ... and it's quite a challenging concept, bringing back that 
vocabulary from previous lessons and then that's the starting off point for this sequence of questioning as it 
really connects those things. 

Interviewer: Do you think with a child, like student A, there is a sense that the visual is actually helping 
send to long-term memory? 

Teacher: I honestly don't think this lesson would have been possible without the interactive 

whiteboard. Because it's a very difficult thing to collaboratively focus upon (a real protractor) because it is 
physically so small and I think this visual of having this angle - and he can see from our previous lessons that 
that's an acute, a right or an obtuse angle, and he's making that connection with the visual. 

Video Clip. In this section of video film where teacher uses the visual large protractor on the screen to 
illustrate the crucial importance of knowing whether we're working with an acute or obtuse angle and 
clarifying the language for those who are struggling. 

Interviewer: So you are drawing and creating angles and stressing the two scales. Why do you have to 
do that? 

Teacher: I think just to make it explicit for them. Also in terms of like their levels of thinking. It has 

gone from the sort of basic levels that, if you're looking at Bloom, like the understanding and the making the 
connections to the applying and then we're going onto the creating and we will be looking at that further. It's 
just about them understanding visually those connections that they have already made verbally but then 
reinforcing that. 

The Video Clip shows the teacher-supporting student T, who is struggling, as a model for developing an 
idea of how to measure the angle on the interactive whiteboard. 

Interviewer: Can you unpack that section because that was quite a lot of questioning? 

Teacher: He is a bit of a struggler in terms of grasping concepts, so it was good for him to be able to 

go through that process, but in terms of like looking at which set of numbers to choose it was about him 
beginning to realize and making connections to-like past lessons and knowledge that he already had, and then 
making new connections to what he is looking at the moment. So in terms of him looking at the top numbers 
we went back to basics and looking at whether it was bigger or smaller than the right angle, and then going 
through the process of discovering - okay we have to look at this set of numbers - and I think that process to 
him would have been a very similar situation for quite a few of the kids in the class, going through the 
process... that step by step, and then essentially thinking that maybe I can ask myself these questions while I 
am doing it. So it's giving them ideas almost like a checklist - what they need to go through. 

Next the Video Clip is about Student A being asked to verbalize this thinking every step of the way. 

Teacher: So he has that very particular need to verbalize. So depending on the individual needs of 

the kids I will really ask them to do different things. 

Interviewer: And he does it very well - why do you think he does it very well? 
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Teacher: Because there's been a lot of lead up and also the previous questioning could have helped 

him as well. 

In this lesson, the IWB is being used, and has become a visual means of inviting the children to share 
their insights. The teacher is insistent that correct language is a key part of the understanding and of sharing 
the concepts, she promotes actively the correct mathematical language. She glories in the achievements of the 
children when their use of the vocabulary indicates their own absorption of the concepts in relating previous 
understanding of the static views of angles as ‘acute’, ‘obtuse’ and ‘right’ to the new dynamic concept of 
measuring angles of rotation. She actively encourages students to verbalize these ideas to their peers. In this 
whole section, the teacher is barely seen on the lesson video because she has handed much of the thinking 
and verbalizing to the students. 

Near the end, as she uses the interactive features to make and measure rotations, she chooses a student 
who normally struggles, to unpack the process of measuring step by step. This was possible because the child 
could speak about each action as he activated the animation on the IWB. The teacher claims that the choice 
of this student was deliberate so that that process to him would have been a very similar situation for quite a 
few of the kids in the class, going through the process... that step by step and then essentially thinking 
‘maybe I can ask myself these questions while I am doing it’. So it's giving them ideas almost like a 
checklist, what they need to go through. She claims she can use such students as models because of the 
support given by the IWB affordances. The sharing with the class through the IWB then aids the rest of the 
class to clarify their understanding. 


5. CONCLUSIONS 

There have been numerous studies lamenting that IWBs have failed to produce the educational change and 
improvements in pedagogy that were expected or hoped for in the first decade of the 21st century. The case 
studies described here are suggesting that important pedagogical change can occur under certain 
circumstances. In particular, it can occur, and is likely to occur, if the teachers already incorporate sharing of 
ideas, creating time to think in dialogue, high level questioning and thinking, and recognition of the 
importance of children owning and sharing their own learning into their teaching 

Analyzing the pedagogical changes that were observed, it is clear that many of them fall into the category 
of dialogic interaction, and thus re-enforce the work of Hennessy. In this study all three teachers stood back 
from the board and invited their student to verbalize while access to the boards has produced regular 
examples of shared cognition, where the insight of a student supports the learning of peers. All three teachers 
constantly ask particular children to model ideas while interacting with the IWB, and all claim they choose 
each child for two purposes: to help the child who is modeling by verbalizing ideas; and to share this with the 
rest of the class, in the belief that children can often explain better to peers than the teacher if they have the 
visual and interactive support of the IWB. 

All three teachers used IWBs to provide waiting time for children to think and verbalize. To most 
teachers this is a difficult area. Waiting time gives opportunity for other children to move off-task. All three 
observed teachers here deliberately used the visual and interactive affordances of the boards to focus and 
engage the class, giving the child doing the modeling, time to think and verbalize. This was seen when 
student T needed time to explain about an aspect of the number on the protractor, and the teacher explains 
their pedagogical thinking: 

He is a bit of a struggler in terms of grasping, concepts so it was good for him to be able to go 
through that process, and I think that process to him would have been a very similar situation for 
quite a few of the kids in the class, going through the process... that step by step... When students 
have time like this, not only can they order their thinking, but they can more effectively share it. 

All these teachers deliberately absented themselves from the IWB. Each teacher ensured students felt they 
had command over the board as the teachers stood to one side. This appears to be allied to these teacher 
beliefs that children must own and share the learning that is taking place, and there was a reluctance to 
dominate, and a deliberate signaling that the child controlled that learning moment. 

It was clear from the observed lessons that all three teachers were skilled questioners. All three used the 
IWB visuals to support open-ended and thinking-rich questioning. However, while IWBs appear to make 
conditions for higher-level questions easier, it is almost certain these three “expert” teachers would use 
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higher level questioning under any teaching conditions. Whenever teachers asked a child to describe or 
explain a set of steps in a process of measuring angles, a quiet ‘why’ was heard as teachers gently interjected. 

So maybe we have been asking the wrong questions. Instead of asking, “what can the special affordances 
of IWBs do to change teaching?” perhaps the question is “what can teachers do to adapt the IWBs for the 
enhancement of good teaching practice?”. By focusing on the natural practices of expert teachers when they 
come into contact with a teaching medium that includes powerful multimedia, a screen large enough to be 
accessible to all students, and interactivity through touch or wireless, we have begun to see in this project 
many examples of enhanced practice. It is clear that in these case studies, dialogic interaction, advanced 
questioning, student ownership of learning and recognition of thinking time are part of the teachers’ common 
practice. However, it is equally clear that when teachers already have these practices embedded in their 
pedagogical armory, they have absorbed the special affordances of the interactive whiteboards into their 
teaching to greatly enhance their teaching effectiveness. 

It is significant that the teachers observed were not necessarily experienced. One was a first year 
probationer. The reason why these three teachers successfully integrated the IWB affordances into their 
teaching did not appear to be connected with experience or age, but rather their willingness to develop an 
environment of shared cognition, deep thinking and high level questioning. It is worth revisiting Salomon, 
Perkins and Globerson’s 1991 proposals that working in partnership with machines can redefine and enhance 
performance. Salomon asks us to consider that perhaps "it is not technology alone affecting minds, but the 
whole ‘cloud of correlated variables’ technology, activity, goal setting, teacher’s role, culture-exerting the 
combined attempt”. 

What this project has observed is indeed precisely that, redefining and enhancement is arising from the 
cloud of correlated variables. It is not, therefore sensible to expect IWBs to transform teaching. The teaching 
comes first. If teachers have the expert pedagogies, IWBs, then the successive waves of current and future 
educational technologies will be successfully incorporated into their growth as teachers and student growth as 
learners. The first year graduate teacher, from the Melbourne Graduate School of Education, at the University 
of Melbourne, has been educated in the Master of Teaching model that develops a practice of clinical 
teaching. The clinical teacher aims ‘to be constantly evaluating a student's learning and progress and 
intervening in specific, targeted ways that are clearly underpinned by research and theory’ (Redman, 2014). 
In conversational style classroom interactions, this young teacher was constantly diagnosing, responding and 
intervening, working towards increasing student understandings. She combines her research-based 
understanding of the difficulties students have in this topic, with her knowledge of her students as learners, 
and utilises the IWBs’ affordances to construct active empowering moments of shared cognition. 
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ABSTRACT 

This study is a review of the creation and evolution of a professional development program modeled on social 
constructivist principles and designed for online educators in a virtual high school who transitioned from the conventional 
classroom to the virtual educational environment. The narrative inquiry focuses on the critical events within the 
professional development program and which were transformational in the process of transitioning from conventional 
brick and mortar instructor to online educators. In addition, the research examined whether or not the social constructivist 
inspired methodology modeled in the professional development transferred into the online teachers’ classroom practice. 
The results of this research have contributed to an understanding of the key components required for the design and 
implementation of a professional development program for teachers who have been trained for conventional classroom 
practice, but who are now called on to teach online in virtual classrooms. 
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1. INTRODUCTION 

Over the course of the last decade, the rapid emergence of virtual schools across the educational landscape 
has not only created a need for a new category of teacher, that is, the online educator, but also it has 
highlighted a pressing need for new models of professional development for educators who are called on to 
teach in online classrooms (Blankenship & Kim, 2012). Ukpokodu (2009) emphasizes the importance of 
effective professional development programs for online teachers since these programs influence the 
pedagogical strategies the teachers will employ in their virtual classes which in turn have an impact on 
student outcomes. An essential starting point in designing a professional development program for online 
instructors is a clear understanding of the knowledge and skills that are specifically associated with effective 
pedagogy in the virtual classroom setting and how these strategies compare to those employed in a 
conventional classroom environment (DiPietro & Sivy, 2011). In terms of professional development in 
general, Blankenship and Kim (2012) clearly demonstrated that traditional models of professional 
development have been ineffective because these models generally responded to institutional requirements 
first without providing authentic and meaningful learning opportunities for the teachers themselves. This is a 
pivotal issue since virtual schools are regularly faced with the challenge of training teachers from the face-to- 
face environment to teach in a virtual classroom environment; however, it is also critical to note that most 
programs do not adequately prepare these teachers as the emphasis in the majority of the professional 
development programs has been upon the use of related technology and not on pedagogy and online 
methodology (Barrett, 2010; Faulkner-Beitzel, 2008). Research has shown that the most effective learning 
within a professional development program for teachers occurs when the learning itself is clearly 
contextualized and when there are social interactions in which authentic practice occurs (Bell & Morris, 
2009). Barrett (2010) makes an important point when he states that the virtual classroom is unlike the 
conventional classroom, and as a result, educators moving into this environment must “unlearn” their 
previous ways of providing instruction and consider effective strategies that both encourage and nurture the 
online student. This notion of a differentiated pedagogy for online delivery is further supported by DiPietro 
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and Sivy (201 1) who call for professional development for prospective online instructors that sensitizes these 
educators to these differences and adequately prepares them for an environment with which most are not 
familiar. 

When educators, who have spent most of their careers in conventional face-to-face classrooms, are 
expected to teach in a virtual school setting, the training and preparation they receive is of critical importance 
since the variables that influence teaching and learning are different for both environments (Wiesenberg & 
Stacey, 2008). Professional development programs for online teachers that help develop more than 
technological proficiency and that focus on teaching strategies which optimize the realities of the virtual 
learning environment can only improve the quality of the educational experience for the online student 
(DiPietro & Sivy, 2011). Smith (2009) also reveals that while some online teachers receive training at the 
university level, most of the training they receive comes from the virtual school in which the teachers work, 
and that generally it is not sufficient, challenging, or ongoing. This narrative inquiry sought to identify the 
particular characteristics of an effective professional development program designed for teachers who arrive 
from the conventional school environment without the specialized skills required for the virtual setting. As a 
result, the problem is that without the proper training and preparation for the virtual classroom, the online 
pedagogy of the virtual school instructors may not adapted to the online setting, and consequently the quality 
of instruction students receive is compromised. 

This particular professional training program was designed with the objective that the participants would 
modify their previous classroom practice and integrate educational practice which is best suited for the 
virtual setting by modeling this practice within the professional development program itself. Narrative 
analysis was employed to uncover common themes or plots in the data and subsequent analysis was carried 
out using a hermeneutic technique for noting underlying patterns across stories of the participants (Webster 
& Mertova, 2007). The narrative analysis identified common critical events which were most meaningful and 
effective for the participants in the professional development program, and which help identify issues, 
challenges and themes that emerge from the professional development program. These critical events were 
recorded and transcribed and given unique identifiers/tags along with a referencing system to allow for 
verification and to provide a means to identify recurring themes/variables that appear within the multiple 
narratives of the participants (L. Webster & Mertova, 2007). Yoder-Wise and Kowalski (2003) suggest that 
this approach helps identify recurring themes, identify consequences, reveal lessons learned, identify what 
succeeded and what did not, describe strengths and vulnerabilities and ultimately help build for future 
experiences, that is, professional development programs for online teachers. Narrative analysis does not 
apply the more traditional approaches toward validity and reliability common to standard qualitative research 
but factors in new measures such as access, honesty, verisimilitude, authenticity, familiarity, transferability, 
and economy (L.Webster & Mertova, 2007). As a result, narrative analysis is a departure from the objectivist 
definition of research validity and reliability (Clandinin, 2013; Webster & Mertova, 2007). The virtual high 
school serves hundreds of students from across the province, has been in existence since 1999 and has a 
number of unique characteristics including its delivery method which is a blended synchronous-asynchronous 
model. This narrative analysis highlights the perspectives of the research participants from the point of their 
inception into the professional training program to the current moment. An examination of a professional 
development program for this group of virtual schoolteachers has relevance for all those charged with the 
design and implementation of training programs for online teachers everywhere. Moreover, this study 
extends the understanding of the role that modeling practice has on the training of online teachers and the 
impact this type of professional development has on an online teacher’s classroom practice. 


2. PROFESSIONAL DEVELOPMENT: ISSUES & CHALLENGES 

In an examination of the Bercow Report which was conducted in 2008 in the United Kingdom and which 
addresses the initial training of teachers, Mroz (2012) highlights many of the obstacles to the general 
improvement of teacher education. The first is that to effect change which is essentially systemic, there needs 
to be a need for a harmonizing of efforts by all involved in the process. This spectrum includes the 
politicians, bureaucrats, administrators, academics, and practitioners who are all involved to varying degrees 
in determining how teachers are prepared for the classroom. Secondly, Mroz argues that funding is required 
to improve teacher education, and at time when the global economy is struggling, monies are not going to be 
allocated for this problem. 
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There is considerable evidence that many professional development programs provided to teachers fail 
because they deliver knowledge and information which is not meaningfully situated or contextualized (Bell 
& Morris, 2009). Effective professional development programs have typically integrated authentic, real- 
world situations which incorporate collaborative work and which are sustained over a period of time 
(Blankenship & Kim, 2012; MacDonald, 2008; Zhang & Watts, 2008). Professional training which is 
specific to online teachers involves a defined set of challenges which range from technology integration to 
the modification of classroom pedagogy to fit the virtual environment which lends itself to the development 
and use of cognitive and social constructivism (Palloff & Pratt, 2011; Powell & Kalina, 2009). Moreover, 
the question of professional development for professionals in the domain of e-learning was explored inside 
the expanding dynamic of the exploding world of online instruction. 

In a study conducted by Korth, Erickson, and Hall (2009), they indicate that a vast majority of teachers 
view themselves as teacher educators, that is, teachers who after a number of years of experience are poised 
to help other teachers in their progression as professional educators. These teacher educators consider that 
their field experiences which they are ready to share can be more purposeful and focused than what teachers 
typically receive from universities or in typical professional development situations (Korth et al., 2009). 
Implicit in this view teachers have of themselves as teacher educators is that professional development must 
be tied to classroom experience that is meaningful and authentic thereby supporting the views expressed by 
Bell & Morris (2009). Kostadinovic (2011) describes professional development in a context where it is 
stated-mandated and primarily driven by universities. It is expected that it be ongoing and formalized and 
inextricably linked to teacher certification though there is little reference in her study as to the efficacy of the 
professional development programs. However, she concludes by stating that teachers must remain current as 
the educational context is constantly evolving and that professional development is at the core of an effective 
educational system (Kostadinovic, 2011). 

Penuel, Fishman, Yamaguchi and Gallagher (2007) conducted a study which explored the effectiveness of 
a professional development program for 454 Science teachers with the goal of identifying which features of 
the program were most effective in implementing change in the classroom practice and aligning the work 
with the prescribed curriculum. Penuel et al. argue that most of the research tied to professional development 
has not been generally useful since much of the data accumulated depended on teacher self-reporting and that 
there was also little objective measure of outcomes especially in the area of curriculum alignment. Similar 
to Kastadinovic, they point out that professional development design has generally been derived from policy- 
maker expectations and did not engage those who would be involved in the actual process. Penuel et al 
designed their study so that the focus was not simply on teacher satisfaction but on actual change in 
classroom practice as well as student outcomes. Referencing previous studies which identified effective 
professional development traits, they designed their program in order that teacher engagement be assured 
from the outset and that the process require active learning by the participants (Penuel, Fishman, Yamaguchi, 
& P. Gallagher, 2007). In addition, they recognized the need to extend the process over a duration of time 
and to establish a climate of collaboration and inquiry which in turn would help establish trust among the 
participants and a sense of mutual support (Suppovitz & Turner, 2000). 

When teachers are involved in professional development which leads to certification, there also seems to 
be a higher level of success as measured by changes in classroom practice (Frye, 2000). Sato, Chung-Wei 
and Darling-Hammond (2008) conducted a three-year study during which they explored the effectiveness of 
a professional development program that was intended to improve teacher assessment skills, especially in the 
area of formative assessment. They found the program to have been successful because the teachers felt the 
program was tied to their classroom realities and the knowledge and skills they developed were both 
meaningful and practical. Similar to Penuel et al. (2007), the teachers in this study found that collaborative 
professional development with a high degree of collegiality not only built trust among the participants, it 
helped them develop a greater sense of self-confidence. In this program, Sato et al. found that active learning 
and inquiry were essential components. 

In studies that describe effective professional development for educators, the recurring notions of 
collegiality and collaboration are highlighted as vital components; however, Stanley (2011) addresses both 
the pitfalls and promises of collaborative teacher groups. Collaboration is understood to be valuable since it 
offers the opportunity to tap into a collective experience and wisdom and simultaneously provides an 
opportunity for reflective practice and inquiry (Stanley, 2011). Professional development that incorporates 
collaboration stands in stark contrast to one-size-fits-all workshops and clinics in which teacher-participants 
are generally passive note-takers. Darling-Hammond, Wei, Andree, Richardson, and Orphanos (2009) found. 
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however, that in general, teachers do not engage in collaborative professional development and that less than 
14% thoughtfully collaborated on curriculum content. They point out how teachers are not trained or skilled 
in collaborative skills. Stanley (2011) argues that the isolated teacher can be replaced by the balkanized 
teacher group set fast against the outside world. She concludes by detailing six elements essential to 
successful teacher collaboration: (a) There must be both individual and group commitment to collaboration, 
(b) There needs to be an understanding of goals and recognition of the tension between knowledge 
acquisition and pedagogical skills, (c) Teacher participation must be active and engaged, (d) There 
necessarily must be within the group a high level of openness and integrity, (e) The teaching assignments of 
the participants must be connected in some fashion, (f) The participants must be ready for new ideas and 
approaches and feel supported in the process. 

Beyond collaboration, effective professional development typically requires active learning and inquiry- 
related activities which engage the participant in acquiring greater content knowledge and pedagogical ( Ali- 
Corlu & Sencer-Corlu, 2012; Penuel et ah, 2007). Ali-Corlu and Sencer-Corlu (2012) describe a program in 
a Turkish university in which teachers were taught scientific inquiry-based skills because it was recognized 
that the students of these teachers would greatly benefit from classroom pedagogy that incorporated the very 
skills the teachers themselves were acquiring. This approach of modeling required that teachers plan, 
prepare, and present an experimental design and in doing so learn, the inquiry concept and then be able to 
teach it effectively. The program included progressively more complex problems that taxed the teachers and 
required them to push beyond their current levels of knowledge and ability. This action-research based 
project demonstrated significant changes in teacher practice but also illustrated the difficulties in training 
teachers’ high-level skills. Corlu & Corlu (2012) conclude with a recurring observation that content mastery 
is, in and of itself, not sufficient to assure positive student outcomes and that there is a balance required 
between content mastery and pedagogical skills. 

2.1 Professional Development and Online Educators 

In an era of regular and ongoing educational reform, the importance of professional development is 
particularly acute (Pallol'f & Pratt, 2005). This issue is accompanied by the need for educational leaders to 
initiate growth and change, to view professional learning in terms which are both personal and engaging for 
the leader and the participants, and to move away from models of professional development which have 
failed to prepare teachers for a new educational reality (Baran, Correia, & Thompson, 2011; Fullan, 2001). 
Fullan (2011) points out the importance of the creation of an environment essential for meaningful and 
sustainable professional development/learning. This position is supported by Webster-Wright (2009) whose 
meta-analysis illustrates that professional development must integrate authentic learning and evaluation 
situations. Webster-Wright (2009) maintains that professionals learn in a fashion which not only shapes their 
practice but that they seek professional training which is inextricably linked to their personal realities. 
Burden (2010) builds on the view of learning as an ongoing process which incorporates the active 
participation and involvement of the professional. Moreover, he considers the significance of the experience, 
the use of reflection, the construction of knowledge, the contextualized nature of learning, and the social and 
collaborative nature of teacher learning. Burden’s research draws on a large body of material on the subject 
of teachers and professional learning; however, there is a particular focus on Web 2.0. He employs this 
corpus to provide a theoretical framework for professional development which is relevant in the current times 
and which integrates tools and learning environments which resonate for educators. Pratt and Palloff (2011) 
view professional development for the online instructor in a manner which is similar to the professional 
learning of any educator; however, they highlight the differences of the online environment and the impact 
and influence of technology. For online instructors, they suggest that professional development must also be 
an organizational endeavor which integrates a systems-wide approach and that effectiveness in the virtual 
environment relies on coordinated team effort. In the context of this narrative inquiry which examines a 
professional development program for online instructors, the process is viewed both from the perspective of 
the teachers involved in the program as well as from the administrators’ organizational perspective. 

In a study conducted by Penuel, Fishman, Yamaguchi, and Gallagher (2007), they undertook a meta- 
analysis of research conducted in the last fifteen years on the topic of professional development for educators 
in general. This study specifically considers those models which have been proven effective as measured by 
the transfer of skills and knowledge acquired by educators in the process of professional development into 
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the actual curriculum in which they teach. In the course of their study, they bring into question many of the 
previous studies which relied primarily on teacher feedback and self-analysis because of the inherent bias and 
lack of detachment noted in the studies. The authors surveyed over 1300 teachers and employed a 
quantitative methodology in order to determine which professional development process was most successful 
in changing teacher practice. They conclude by stating that any professional development program which is 
sustained and communal tends to have the most impact. Communal also includes online communities of 
practice which tend to reinforce the strengths of online learning and simulaneously illustrate the inherent 
challenges (Cook, 2007; Zhang & Watts, 2008). These studies underscore the complexities of professional 
development for educators and demonstrate that more research is required in this field in general and more 
specifically in the area of e-learning. 

Smith (2009) examined a number of studies dealing with online teaching standards, the perceptions of 
online teachers, and national standards for online education. His conclusions are significant as he argues first 
that standards for online teachers are not truly reflective of the roles that online teachers must perform and 
secondly that most online teachers are themselves not even aware of any standards which apply to online 
teaching. In this study. Smith also points out that online education is offered using different platforms and 
delivery modes. These, in turn, require different training and offer different pedagogical options which 
implies that professional development for online teachers needs to be improved. 

In a survey conducted by Rice and Dawley (2009) of 259 online teachers, they highlight many of the 
same deficiencies identified in earlier studies by D’Alba and Sandberg (2006). There is little empirical 
evidence of effectiveness and all too often professional development programs fail to consider the importance 
of contextualized training versus the acquisition of skills and knowledge which does reflect the realities of 
the teacher involved. Rice and Dawley cite the rapidly evolving nature of technology as one key issue in 
professional development programs for online teachers, but a significant proportion of online teachers 
involved in this survey indicated that they received either no training prior to teaching online, or they stated 
that their training was inadequate. Moreover the teachers involved in Rice and Dawley’s survey described 
the topics involved in their professional development as those which covered technology usage, lesson 
design, facilitation strategies, and foundational knowledge. Ultimately, pragmatic issues drive the type of 
training online teachers receive, but as Rice and Dawley conclude more must be given over to what 
constitutes best educational practice in the virtual school setting. 

What constitutes effective professional development for online teachers is not very easy to demonstrate as 
evidenced by the study conducted by Hathaway and Norton (2012). The initial premise put forth by the 
authors is that there are aspects of online teaching which are significantly different from conventional brick 
and mortar classroom instruction, and for this very reason, those teaching online require a robust, ongoing, 
and effective professional development program. As indicated by Archambault and Crippen (2009), the vast 
majority of online teachers have been recruited from the conventional classroom, and most are identified as 
highly effective teachers whose skills can and should transfer well into the virtual classroom. Hathaway and 
Norton examined two programs used to prepare classroom teachers for the online environment. In essence, 
one program was a light program which provided only the basics with an emphasis on the technology, while 
the second program covered a wide range of topics relevant to the online classroom. What the authors had 
anticipated was that those who had received indepth preparation would be more comfortable in the online 
classroom and they would report greater success when compared to those who had received only minimal 
training. The results surprised the authors as this correlation could not be drawn suggesting that classroom 
teachers who mastered their subject content and who could communicate effectively would transition 
comfortably into the virtual environment and that additional training was not as essential as suggested by 
others. Hathaway and Norton contextualized their findings by stating that much of the data came from the 
online teachers themselves in self-reporting and that their respective interpretations likely varied 
considerably. 

When preparing educators to teach in the virtual classroom, modeling online instruction for prospective 
online teachers is a productive and effective means of introducing the technology, the tools at the teachers’ 
disposal, the challenges and the potential that this educational environment provides (Bell & Morris, 2009; 
Cook, 2007). Beyond the value of acquiring the relevant competencies to teach online, prospective 
instructors engaged in effective professional development, according to Cook (2009), also have the 
opportunity to live the experience as learners and to understand the situation from the learners’ perspective. 
In this way their knowledge and understanding is constructed collaboratively and in a manner which typifies 
the basic tenets of social constructivism. 
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2.2 Social Constructivism 

In the context of this case narrative inquiry, Vygotsky’s (1978) principles are particularly relevant as the 
professional development must be experienced as authentic and meaningful given that change occurs most 
effectively when it is the by-product of human interaction and exchange (Andrews, 2012). 

Sutinen (2008) situates social constructivism as a theory of learning within the larger framework of 
educational theorists ranging from Dewey to Piaget and Vygotsky. Sutinen (2008) maintains that 
constructivism is based on the assumption that the individual’s construction processes cannot be influenced 
from the outside. Furthermore, constructivist theories of learning often rely on a notion of development in 
which it is assumed that the individual’s learning process will develop following an intuitive and native 
capacity which takes into consideration an individual’s place in the world and that individual’s interaction 
with everyone and everything. Moreover, Sutinen (2008) points how learning theories by Dewey and others 
are frequently embedded with social constructivist concepts and modalities that situate the learner in a 
context where there is constant and ongoing interaction. Consequently, and in the context of the professional 
development program, the narrative inquiry describes learning as an important collaborative and social 
process for the online teachers involved. 

2.3 Study Group 

The virtual school staff involved in the study was located in Shawville, Quebec. The staff included 8 teachers 
and two school administrators. The virtual school delivers a high school curriculum to students across 
Quebec using teachers who were selected from the regional school boards because of their subject mastery as 
well as their willingness to tackle the challenges of a synchronous-asynchronous blended delivery model. 
Initially, and as expected, their methodology reflected conventional brick and mortar styles. It became 
quickly apparent that this approach did not engage the students and did not optimize the potential of the 
virtual setting. The professional training program in which they embarked was intended to model what was to 
be expected of them in the online classes and to parallel the provincial curriculum which is grounded in 
social constructivist principles. 


3. CONCLUSION 

Research suggests that professional development should be a continuous process which allows teachers time 
to practice, to provide modeling of instruction, and to offer feedback (McMaster & Tschannen-Moran, 2009). 
Moreover in the field of e-learning, professional development functions as the path for developing faculty 
members to be effective online and to increase student outcomes (Keengwe, Kidd, & Kyei-Blankson, 2009). 
Currently there is an urgent need for effective professional development for educators in virtual schools; 
however, there is simultaneously a dearth of research on professional development models for the many 
forms e-learning may take. 
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ABSTRACT 

E-learning systems, or learning management systems, as it is known in the field, sit at the heart of educational systems 
and are used to systematically deliver on-line content and facilitate the learning experience around that content. It 
becomes essential to ensure that Learning Management Systems of a high standard are being developed. In the field of 
Systems Development Methodologies (SDMs), much research has been done and the value of SDMs is proven and 
documented. To ensure that e-leaming systems of outstanding quality are being developed, it is crucial that systems 
development methodologies are being used as they can have a significant impact on the development process. 

The main focus of this study was to determine if systems development methodologies are being utilised in the 
development of electronic learning systems in South Africa and if these methodologies are being applied effectively. By 
utilising a survey as the main research method and statistically analysing the dataset, meaningful results were obtained. 
Some of the findings were that academic institutions are clearly using open-source LMSs more than proprietary LMSs. 
The LMS platform had a moderate effect on the success of the LMS. Open-source LMS users were very satisfied with 
their LMS platform and not inclined to consider a proprietary LMS for future projects. Proprietary LMS users may be 
inclined to consider an open-source LMS for future projects. It was found that a very strong relationship exists between 
respondents that indicated that they use RAD as SDM and the impact this SDM had on the quality and productivity of the 
development process. It was found that a very strong relationship exists between the perceived support that the SDM 
provides and the impact of the SDM on the quality and productivity of the development process. This in turn has a strong 
influence on the impact systems development methodologies have on the quality of learning management systems. 


KEYWORDS 

e-learning, learning management systems, systems development methodologies. 


1. INTRODUCTION AND PROBLEM STATEMENT 

With the dramatic increase in the popularity and use of e-learning and e-learning systems, it becomes a 
necessity to ensure that systems of a high quality are being developed. If e-learning is applied efficiently, it 
can go a long way towards helping learners to generate and obtain knowledge for themselves. Many cultures 
around the world still have no access to even basic education. E-learning can be one of the answers to this 
problem. 

Even a cursory study into e-learning makes it clear that there is still plenty of confusion surrounding e- 
learning and the technology and applications involved with it. It becomes apparent there is still much to 
benefit, from further research into this field. 

In another and not unrelated field of Systems Development Methodologies (SDMs), much more research 
has been done and the value of SDMs is proven and documented. However, there is still a lack of empirical 
evidence on the actual use and effectiveness of SDMs and even more so with its use and effectiveness in the 
development of e-learning systems. 

SDMs have had a major impact on the development of software systems over the last 40 odd years and 
are an indispensable tool for developing systems in recent times. E-learning systems, being an emerging 
subset in software systems technology, could well profit from the benefits SDMs have to offer. 

The aim of this study will be to research the use and effectiveness of SDMs in the development of e- 
learning systems. 
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2. LITERATURE STUDY 

2.1 Terminology 

For the purposes of this study e-learning can be seen as a medium for delivering and facilitating learning, 
through electronic means. 

The term Learning Management System will also be used for the purposes of this study. An LMS is a 
software system that is used to systematically deliver content on-line and to facilitate the learning experience 
around that content. 

The following definition of a systems development methodology will be used (Huisman, 1999): 

• Systems development approach 

This can be defined as the philosophical view on which the methodology is built. Thus, the set of 
goals, guiding principles and beliefs, basic concepts and principles of the systems development 
process that drive interpretations and actions in systems development (Iivari et al., 1998; Iivari el 
al., 1999). Examples of systems development approaches are the structured approach, object- 
oriented approach, information modelling, etc. 

• Systems development process model 

Wynekoop and Russo (1993) define a process model as a representation of the sequences of 
stages through which a system evolves. Some examples of process models are the linear life-cycle 
model, the spiral model and incremental model. 

• Systems development method 

A method is a systematic approach to conducting at least one complete phase of systems 
development, consisting of a set of guidelines, activities, techniques and tools, based on a particular 
philosophy of systems development and the target system (Wynekoop & Russo, 1993). Examples 
include IE, SSADM, etc. 

• Systems development technique 

A systems development technique consists of a well-defined sequence of actions, ensuring 
successful results if used correctly (Iivari et al., 2000; Brinkkemper, 1996), for example entity 
relationship diagrams and data flow diagrams.. 

2.2 The Use of Systems Development Methodologies 

It is argued that the use of SDMs will aid in the development process by making it more effective, secure, 
predictable and easier to control, whilst improving productivity and quality (Fitzgerald et al., 2002). Whether 
systems development methodologies are being used in practice has been a point of interest for academics for 
an extended period of time. 

Russo et al. (1995) found that only 6% of developers followed an SDM rigorously. Chatzglou and 
Macaully (1996) found that 47% of their population had never used a methodology. Avison and Fitzgerald 
(2003) also found this number to be low. Less than 50% of organisations follow SDMs strictly (Glass, 1999). 
Huisman and Iivari (2006) determined that when SDMs are being used in practice, it is not to the full extent. 
Fitzgerald et al. (2002) show that the use of formalised SDMs was significantly higher in larger organisations 
(more than 1 000 employees) and larger IS departments (more than 20 personnel). In recent studies 
conducted by Brits (2011) and Janse van Rensburg (2012) it was found that 74% of organisations in South 
Africa make use of SDMs. Conradie (2010) and Wagener (2012) also found this number to be above 75%. 

2.3 The Effectiveness of Systems Development Methodologies 

Empirical research on the effectiveness of SDMs is very limited, which seems to be due to the lack of 
standard criteria that measures SDM effectiveness (Conradie 2010). This section will confer the results on the 
effectiveness of SDMs that were found in the literature. 

Although many SDMs have been successfully utilised over the years, there have been many software 
failures, which has questioned the relevance of SDMs. Even with the use of SDMs, projects are still being 
abandoned halfway, still overshooting the budget, and still not being delivered within an appropriate time 
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frame (Truex et al., 2000; Gruner et al., 2007). However, great strides have been made in the last four 
decades; yet there is still much to be learned from studies into the effectiveness of SDMs. 

There are several compelling reasons for using SDMs. Fitzgerald et al. (2002) state that systems 
development methodologies have been endorsed by the literature, as being capable of ensuing that the 
development process is more effective, secure, predictable and easier to control. Gruner et al. (2007) argue 
that it is challenging and precarious to develop software without the guidance or structured process that an 
SDM can provide. Huisman and Iiavari (2006) reasoned that SDMs introduce structure to the design process, 
thereby improving the effectiveness of software development and thus ensuing more consistent outcomes. 
The use of software methodologies is said to decrease the risk of failure of an information system (IS) project 
(Hull et al, 2002; Avison & Fitzgerald, 2003). 


3. DATA COLLECTION AND ANALYSIS 

In the field of information systems, surveys are a popular strategy to employ in the collection of empirical 
evidence. A questionnaire, as the measurement element, was developed in collaboration with a statistical 
consultation service. The constructs, identified from the conceptual research model, were operationalised by 
selecting measurement scale items (questions) and scale types. The questions were adapted from previous 
research studies, which proved to be reliable. The questionnaire was concise and relevant and contained 
mostly leading, importance and 5-point Likert scale questions. It was distributed electronically, as a macro- 
enabled Excel file, to personnel at institutions of higher education in South Africa that are responsible for 
developing and/or deploying e-learning systems. Software companies in South Africa, which develop e- 
learning solutions, were also targeted. The questionnaire had extensive built-in entry validation to ensure that 
the respondents fill in the correct values as well as coding to assist in data analysis. 

Fifty responses were received from a possible one hundred and twenty-five responses. Therefore, the 
participation rate equalled 40% with 50 cases available for data analysis. 


4. RESULTS OF THE RESEARH QUESTIONS 

The relationship between industry and platform of the LMS (RQ1): 

In academic institutions open-source LMSs are being used in almost 61% of the cases and in the private 
sector just above 52%. Many of the academic institutions indicated that they are in the process of moving 
towards an open-source learning management system but that they are currently still using a proprietary 
LMS. 

The relationship between industry and number of students (RQ2): 

Crosstab analysis was done to determine if the type of industry relates to the number of users of the 
specific LMS in question. The results of the crosstab analysis can be seen in Table 1 and it was statistically 
significant with p < 0.01 and a Cramer’s V of 0.568. 

It is quite noticeable that 75% of the academic institutions’ LMSs are being utilised for 10 000 or more 
students, learners or employees. The private sector almost exclusively uses LMSs with more than a 1 000 
users. 


Table 1 . Crosstab analysis: Research Question 2 


Industry 

1-99 

100-499 

Number of students/learners/employees using the LMS 
500-999 1000-4999 5000- 9999 10 000 or more 

Total 

Academic 

7.1% 

10.7% 

0% 

0% 

7.1% 

75% 

100.0% 

Private 

Sector 

0% 

15.8% 

0% 

21.1% 

31.6% 

31.6% 

100.0% 

Total 

4.3% 

12.8% 

0% 

8.5% 

17% 

57.4% 

100.0% 
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The relationship between industry and method of procurement of LMS (RQ3): 

Crosstab analysis was to determine if the type of industry relates to the LMS platform being used. The 
results of the crosstab analysis can be seen in Table 2 and it was statistically significant with p < 0.05 and a 
Cramer’s V of 0.447. There is very little in-house development being done when it comes to the procurement 
of LMSs with a combined (academic and private sector) total of 17.4% of the respondents indicating that 
they make use of in-house development. With 32.6% of the procurement of LMSs being done by adapting 
open-source LMSs, it is by far the preferred method of procuring an LMS. To elaborate on the 
abovementioned adaption of open-source LMS, it is worth noting that 39.3% of academic institutions use 
open-source systems “as is”, while only 5.6% of the private sector use LMSs “as is”. Only 21.4% of 
academic institutions adapt their LMSs compared to the 50% of the private sector. A combined total of 
23.9% of the procurement of LMSs are being done by purchasing off the shelf products. 


Table 2. Crosstab analysis: Research Question 3 





Procurement of LMSs 


Industry 

Purchased 

In-house 

Open-source 
(used as-is) 

Adapted from open- 
source system 

Total 

Academic 

25% 

14.3% 

39.3% 

21.4% 

100.0% 

Private 

Sector 

22.2% 

22.2% 

5.6% 

50% 

100.0% 

Total 

23.9% 

17.4% 

26.1% 

32.6% 

100.0% 


The relationship between the LMS platform and perceived success of the LMS (RQ4): 


T-tests indicated that the LMS platform had a medium effect on the success of the LMS with an effect 
size [f ~ ) of 0.34 and a statistical significance, p > 0.05. 

The relationship between the LMS platform and the satisfaction with the platform (RQ5): 

T-tests indicated that the platform of the LMS has a very high effect on the inclination of respondents to 
consider using an open-source LMS for future projects with an effect size (L) of 1.7 1 and it is statistically 
significant with p < 0.001. Respondents that use an open-source LMS would be inclined to keep using an 
open-source LMS for future projects with a mean value of 4.4. 1 Respondents that use a proprietary LMS will 
be less inclined to consider an open-source LMS for future projects with a mean value of 2.75. T-tests also 
indicated that the platform of the LMS has a very high effect on the inclination of respondents to consider 

using a proprietary LMS for future projects with an effect size (f 2 ) of 1.04 and it is statistically significant 
with p < 0.001. Respondents that use a proprietary LMS would be inclined to keep using a proprietary LMS 
for future projects with a mean value of 3.65. Respondents that use an open-source LMS will be less inclined 
to consider a proprietary LMS for future projects with a mean value of 2.4. 

The relationship between the procurement method of the LMS and the satisfaction with 
the platform (RQ6): 

T-tests indicated that the procurement method of the LMS has a very high effect on the inclination of 
respondents to consider using an open-source LMS for future projects. Respondents that use an open-source 
LMS would be inclined to consider using an open-source LMS (as is, i.e. not adapting it) for future projects 

with an effect size (/ ) of 2.16 and it is statistically significant with p < 0.001. Respondents that use a 
proprietary LMS would be inclined to consider using an open-source LMS (as is, i.e. not adapting it) for 
future projects with an effect size (/~) of 2.11 and it is statistically significant with p < 0.001. Respondents 
that use an open-source LMS would also be inclined to consider adapting an open-source LMS for future 
projects with an effect size (/ ) of 1.50 and it is statistically significant with p < 0.001. Respondents that 


1 All the means are indicated as a value out of a possible 5, as a five-point Likert scale was used. 
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use a proprietary LMS would also be inclined to consider adapting an open-source LMS for future projects 
with an effect size (/ ' ) of 1.45 and it is statistically significant with p < 0.001. 

It is clear that the respondents will consider using open-source LMSs for future projects, but they are less 
inclined to adapt such LMSs, than using them as is. 

The relationship between the perceived success of the LMS and the satisfaction of the 
platform (RQ7): 

There is a strong relationship that exists between the success of the LMS and the satisfaction of the 
platform used with a Spearman’s rho of 0.589 and it is statistically significant with p < 0.001. 


The relationship between the type of SDM used and the success of the LMS (RQ8): 

T-tests indicated that the use of formal SDMs had a medium effect on the success of the LMS with an 
effect size (/' ) of 0.34 and a statistical significance, p < 0.05. 


The relationship between the type of SDM used and the impact of the SDM on the 
development process (RQ9): 

This research question determines if there is a relationship between the type of formal standard SDM used 
and the impact of the SDM on the quality and productivity of the development process. There is a very strong 
relationship that exists between respondents that indicated that they use RAD as SDM and the impact this 
SDM had on the quality and productivity of the development process. Spearman’s rho = 0.618. p < 0.001. 
There is a strong relationship that exists between respondents that indicated that they use IE as SDM and the 
impact this SDM had on the quality and productivity of the development process. Spearman’s rho = 0.517. p 
< 0.01. The SDM with the strongest influence on the quality and productivity of the development process is 
RAD, which is an Agile methodology. 

The relationship between the performance expectancy of the SDM and the impact of the 
SDM on the quality and productivity of the development process (RQIO): 

There is a strong relationship that exists between the performance expectancy of the SDM and the impact 
of the SDM on the quality and productivity of the development process with a Spearman’s rho of 0.563 and it 
is statistically significant with p < 0.01. 

The relationship between the performance expectancy of the SDM and the perceived 
impact of the SDM on the quality of the LMS (RQll): 

There is a strong relationship that exists between the performance expectancy of the SDM and the 
perceived impact of the SDM on the quality of the LMS with a Spearman’s rho of 0.547 and it is statistically 
significant with p < 0.01. 

The relationship between the perceived support that the SDM provides and the impact of 
the SDM on the quality and productivity of the development process (RQ12): 

There is a very strong relationship that exists between the perceived support that the SDM provides and 
the impact of the SDM on the quality and productivity of the development process with a Spearman’s rho of 
0.744 and it is statistically significant with p < 0.001. 

The relationship between the perceived support that the SDM provides and the perceived 
impact of the SDM on the quality of the LMS (RQ13): 

There is a strong relationship that exists between the perceived support that the SDM provides and the 
perceived impact of the SDM on the quality of the LMS with a Spearman’s rho of 0.541 and it is statistically 
significant with p < 0.01. 
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The relationship between the impact of the SDM on the quality and productivity of the 
development process and the success of the LMS (RQ14): 

There is a weak relationship that exists between the impact of the SDM on the quality and productivity of 
the development process and the success of the LMS with a Spearman’s rho of 0.177 and it is not statistically 
significant with p > 0.05. 

The relationship between the impact of the SDM on the quality and productivity of the 
development process and the perceived impact of the SDM on the quality of the LMS 
(RQ15): 

There is a strong relationship that exists between the impact of the SDM on the quality and productivity 
of the development process and the perceived impact of the SDM on the quality of the LMS with a 
Spearman’s rho of 0.543 and it is statistically significant with p < 0.01. 

The relationship between the perceived impact of the SDM on the quality of the LMS and 
the success of the LMS (RQ16): 

There is a moderate relationship that exists between the perceived impact of the SDM on the quality of 
the LMS and the success of the LMS with a Spearman’s rho of 0.376 and it is statistically significant with p 
<0.05. 

The need to design an SDM specific to LMS and the SDM that was used to design the 
current LMS (RQ17): 

There is a strong negative relationship that exists between respondents that indicated that they use another 
(not listed) SDM and on opinion that there is room for a newly designed SDM specifically for developing 
LMSs with a Spearman’s rho = -0.402 and a statistical significance, p < 0.05. There is a strong relationship 
that exists between respondents that indicated that they use another (not listed) SDM and on the opinion that 
an existing SDM can be adequately adopted to suit the needs of e-learning with a Spearman’s rho = 0.437 
and a statistical significance, p < 0.01. There is a moderate negative relationship that exists between the 
respondent that indicated that they use IE (Information Engineering) as SDM and the opinion that an existing 
SDM can be adequately adopted to suit the needs of e-learning with a Spearman’s rho = -0.399 and a 
statistical significance, p < 0.05. 

Summary of the results 

A summary of the statistical analyses that were performed, and the relationships that were evaluated, can 
be seen in table 3. 


Table 3. Research questions as tested (Note: p = Level of Significance. ***=<0.001, **=<0.01, *= <0.05, ‘=<0.1; 

n/a=this type of test could not/was not performed) 


Research 

Question 

Path tested 

Spearman’s rho 
(p) / Cramer’s 

vO) 

Effect Size 

(L) 

Relationship / 
Correlation / Level 
of Association 

RQ1 

Industry — > Platform Used 

(p c = 0.709*** 

n/a 

Very strong 

RQ2 

Industry — > Number Users 

cp c = 0.568** 

n/a 

Very strong 

RQ3 

Industry — * Procurement LMS 

(p c = 0.447* 

n/a 

Very strong 

RQ4 

Platform_Used — » Success LMS 

n/a 

0.34 

Medium effect 

RQ5 

Platform_Used — > 

n/a 

j 72*** 

Very high effect 


Satisfaction_LMS_Platform 


2 04*** 


RQ 6 

Procurement LMS — » 

n/a 

2 22 *** 

Very high effect 


Satisfaction_LMS_Platform 


2.16*** 





1.50*** 





2 45 *** 


RQ7 

Success LMS — > 

p = 0.589*** 

n/a 

Strong 


Satisfaction_LMS_Platform 




RQ 8 

Fonnal SDM Use — > Success LMS 

n/a 

0.34* 

Moderate 
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Research 

Question 

Path tested 

Spearman’s rho 
(p) / Cramer’s 

vO) 

Effect Size 

(D 

Relationship / 
Correlation / Level 
of Association 

RQ9 

Extent_Standard_SDM (RAD) — > 
Impact_SDM_Quality_Productivity 

p =0.618*** 

n/a 

Very strong 

RQ10 

Performance Expectancy SDM — » 
Impact_SDM_Quality_Productivity 

p = 0.563** 

n/a 

Strong 

RQ11 

Perfonnance Expectancy SDM — * 
Impact_Quality_LMS 

p =0.547** 

n/a 

Strong 

RQ12 

Perceived Support SDM — > 
Impact_SDM_Quality_Productivity 

p = 0.744*** 

n/a 

Very strong 

RQ13 

Perceived Support SDM — > 
Impact_Quality_LMS 

p = 0.541** 

n/a 

Very strong 

RQ14 

Impact SDM Quality Productivity — * 
Success LMS 

p = 0.177’ 

n/a 

Weak 

RQ15 

Impact SDM Quality Productivity — » 
Impact_Quality_LMS 

p =0.543** 

n/a 

Strong 

RQ16 

Impact_Quality_LMS — » Success LMS 

p = 0.376* 

n/a 

Moderate 

RQ17 

Need_SDM_LMS -» 
Extent_Standard_SDM (Other) 

p =0.437** 

n/a 

Strong 


5. CONCLUSION 

The inferential statistics that are presented in this paper reveal some interesting facts about the relationships 
between certain constructs related to learning management systems and systems development methodologies 
in South Africa. 

Crosstab analysis indicated that open-source learning management systems are preferred to proprietary 
LMSs, especially in the academic sector. It also revealed that the majority of LMSs in South Africa are being 
used for 10 000 or more users. Academic institutions also tend to use open-source LMSs as is and the private 
sector adapts open-source LMSs to a large extent. 

There is a strong relationship that exists between the perceived success of the LMS and the satisfaction of 
the platform used. It makes sense that respondents will be satisfied with their LMS platform if the LMS is 
successful. In general, respondents are very satisfied with their current LMS platform. 

Open-source LMS users are not inclined to consider a proprietary LMS for future projects. Proprietary 
LMS users are more inclined to consider an open-source LMS for future projects. Both proprietary and open- 
source LMS users will be less inclined to adapt an open-source LMS, than using it as is. 

A very strong relationship were found between the perceived support that the SDM provides and the 
impact of the SDM on the quality and productivity of the development process. There is a very strong 
relationship that exists between respondents that indicated that they use RAD as SDM and the impact this 
SDM had on the quality and productivity of the development process. This correlates with what the literature 
revealed on the use of Agile methodologies in the post-methodology era. 

There is also a strong relationship that exists between the performance that users expect SDMs can 
provide and the impact that the specific SDMs have on the quality and productivity of the development 
process. A very strong relationship exists between the support that the SDM provides in the development 
process and the actual impact that the SDM has on the quality and productivity of the development process. 

Both the perceived support that the SDM provides and the impact of the SDM on the quality and 
productivity of the development process have a strong influence on the impact the SDM has on the quality of 
the LMS. 

The study made a contribution to the discipline of information systems, and more specifically, learning 
management systems, by providing insights with regard to the factors effecting the use and effectiveness of 
SDMs in developing LMSs. As far as could be ascertained, this study generated the first empirical data on the 
procurement and development of LMSs in South Africa. A holistic picture was drawn on what the South 
African e-learning market looks like and it was determined that the extent of use of open-source LMSs 
exceeds what was initially believed. 
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Almost 75% of the respondents indicated that there may be room for a newly designed SDM specifically 
for the development of LMSs and tools for LMSs. This could well be a great opportunity for future research 
in this field. This study may also serve as a stimulus for future research in the field of learning management 
systems and more specifically the development of LMSs by using systems development methodologies to 
enhance the chances of success for those systems. 
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ABSTRACT 

In a hospital, clinical teachers must also care for patients, so there is less time for the teaching of clinical courses, or for 
discussing clinical cases with interns. However, electronic learning (e-learning) can complement clinical skills education 
for interns in a blended-learning process. Students discuss and interact with classmates in an e-learning collaborative 
environment. E-learning can assist clinical training and provides a collaborative environment, but every student has 
individual learning preferences on the e-learning platform. A typical platform, such as a learning management system 
(LMS) does not provide individual learning activities for every student. This paper clusters students into two groups: 
active and inactive groups. In each group, students’ learning behavior patterns, i.e., the association rules for activities, are 
derived from the transaction data for the LMS. The cluster to which a student belongs defines the online learning 
behaviors, from the activity association rules. The method then provides individual preferred activities. Teachers instruct 
students in accordance with their aptitude, as derived from the learning behavior pattern. The cluster analysis shows that 
students in active group often view teaching videos after completing feedback. Students in the inactive group often view 
teaching materials after adding posts on a forum. 


KEYWORDS 

Internship, clinical skill education, e-learning, blended learning, clustering, association rules 


1. INTRODUCTION 

In a hospital, clinical teachers must devote time to patients, so they have less time to teach clinical courses, or 
to discuss clinical cases with interns (Prideaux et al., 2000, Ramani and Leinster, 2008). However, e-learning 
can be a complement to clinical skill education for interns (Ruiz et al., 2006). Most medical students feel that 
e-learning has a positive impact on the acquisition of clinical skill and knowledge. It is an integrated, blended 
approach (Gormley et al., 2009). Students who use an e-learning platform as a complement to in-classroom 
education obtain higher scores in the final examination (Seluakumaran et al., 2011). Blended learning, which 
is clinical learning combined with e-learning, results in a greater acquisition of knowledge in radiology 
internship (Mahnken et al., 2011). An e-learning course in radiology gives greater knowledge acquisition 
than a standard lecture-based course and is a cost-effective alternative to standard lecture-based teaching 
(Hadley et al., 2010). 

Medical education in radiology includes undergraduate, postgraduate and continuing education. The 
existing e-learning platforms for radiology are on-line neuroradiology education resources (NeuroRAD) 
(Sparacia et al., 2007), the American Association of Physics in Medicine (AAPM) and the Radiological 
Society of North America (RSNA) (Brambilla et al., 2011). These platforms were developed for physicists 
and radiologists in continuing education, but few platforms have been developed for undergraduate radiology 
internship. Radiology clinical training includes image diagnosis, nuclear medicine, radiation oncology, 
quality assurance and safety examination, which require discussion and interaction with a teacher and 
classmates. In an e-learning collaborative environment, radiology interns can interact with their classmates at 
anytime, to construct their own knowledge base(Brambilla et al., 2011). Clinical teachers provide the 
radiological content, such as image banks and special clinical cases (Perriss et al., 2006). Radiology interns in 


189 



ISBN: 978-989-8704-08-5 © 2014 


different areas discuss their cases with others by posting on a forum. This study establishes an e-learning 
platform to assist clinical skill training. 

E-learning can assist clinical training and provides a collaborative environment, but every student has 
individual learning preferences on the e-learning platform. A typical platform, such as the learning 
management system (LMS), does not provide individual learning activities for every student. In order to 
provide an individual learning environment, data mining technology is applied to educational systems 
(Romero and Ventura, 2007). This paper clusters students into two groups, based on their activity 
preferences: active and inactive groups. Students in the active group, often view the course, complete 
feedback, add posts, update posts and view discussions on the forum. All of the activity association rules for 
the two groups were retrieved, in order to determine students’ learning behavior patterns, for each cluster. 
For example, active group students often view teaching videos for the lessons, after viewing the course and 
completing feedback. They also prefer to write messages to other classmates to communicate emotions. 

The remainder of this paper is organized as follows. Section 2 discusses related studies. Section 3 
describes the proposed method. Section 4 presents the experimental results. Section 5 summarizes the 
findings, states the limitations of this study. 


2. RELATED STUDIES 

Two data mining techniques are used: clustering technique and association rules. Some data mining 
applications for e-learning are also described. 

2.1 Clustering Technique 

Clustering techniques, which are usually used to segment markets (Punj and Stewart, 1983), seek to 
maximize the variance between groups and to minimize the variance within groups. A number of clustering 
algorithms have been developed, such as K-means, hierarchical and fuzzy c-means algorithms (Omran et ah, 
2007). K-means clustering (MacQueen, 1967) is a widely used similarity grouping method that partitions a 
dataset into k groups. The K-means algorithm assigns instances to clusters, based on the minimum distance 
principle. An instance is assigned to a cluster based on the minimum distance to the center of the cluster, over 
all of the k clusters. 

2.2 Association Rules 

Association rule mining determines the associations between two sets of products in a transaction database. 
Agrawal et al. (1993) formalized the problem of determining association rules that satisfy the minimum 
support and the minimum confidence requirements. For example, if a set of purchase transactions includes a 
set of product items I, an association rule is an implication of the form, X => Y, where X cl, Y cl, and X D 
Y = O. X is the antecedent (body) and Y is the consequence (head) of the rule. Two measures, support and 
confidence, are used to determine the quality of an association rule. The support of a rule is the percentage of 
transactions that contain both X and Y and the confidence of a rule is, the fraction of all transactions that 
contain X that also contain Y. 

2.3 Data Mining Applications for E-learning 

Data mining technologies include on-line analytical processing (OFAP), clustering, association rules and 
classification and visualization (Zalane, 2002, Talavera and Gaudioso, 2004, Zorrilla et al., 2005, Mostow 
and Beck, 2006, Romero and Ventura, 2007, Romero et al., 2008). For example, Zorrilla et al. (2005) built a 
web log data cube for OFAP operation, to analyze the log to obtain the information that allows teachers to 
evaluate the learning process. Talavera and Gaudioso (2004) clustered users into groups, to determine their 
behavior patterns and evaluations. Mostow and Beck (2006) developed a listening tool that uses visualization 
technology to help children to decode words and comprehend stories. Zalane (2002) used association rules 
that use a learner’s access history to recommend on-line learning activities, or shortcuts on a course website. 
Romero et al. (2008) used clustering, association rules and classification technologies to discover knowledge 
from a learning content management system. 
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3. METHODOLOGY 


This section proposes a method to understand learning behavior patterns for interns, which is shown in 
Figure 1. The activity usage count for each student on LMS was calculated. The missing values were filled 
using proper data preprocessing. Because some activities are used by few or no students, feature selection 
was applied to some activities. These activities were not taken into account. The continuous values for the 
activity usage count were then transformed into discrete preference values, to form a student-activity 
preference matrix. 

The K-means clustering method was then used to cluster students into activity preference groups, based 
on the similarity between students’ activity preferences, which were measured using Pearson’s correlation 
coefficient, as shown in Eq.(l). The r\F and denote the average rating score of all activities used by 
students Sj and Sj respectively. The variable I denotes the mix of the set of activities. The r s . A and r s . A 


denote the rating score that students S; and Sj used activity A. 

S A el(rs i ,A-r5i)(rs j ,A-riy) 


corr(Sj, Sj) = 


ZAel( rSiA" 1 ^) SAel( r Sj,A — r Sj) 


( 1 ) 


In each group, students’ online learning behavior pattern, the association rules, were derived from the 
transaction data of the learning management system (LMS). The cluster to which a student belongs defines 
the online learning behavior pattern, based on the association rules in that cluster. The method shows the 
preferred activities for every student. 



Figure 1. An overview of the proposed method 
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3.1 Data Preprocessing 

The students and activities transaction data from the learning management system (LMS) were firstly pre- 
processed. The linear method was used to fill the missing values. Feature selection was then applied. There 
are many learning activities on the learning management system, but some activities are never or seldom 
used. The number of the selected activities is 18. They include viewing the course, folder, discussion, 
resource, video, completing feedback, adding and updating posts on a forum, writing messages and updating 
user profiles. 

3.2 Activity Preference Scores of Students 

The usage counts for the selected activities are continuous values. Eq. (2) (Lin et al., 2003) is used to 
transform the continuous values to discrete values, i.e., -1, 0, or 1. 

Z = ^ (2) 

°X 

where X is the activity usage count, x and ct x are, respectively, the mean value and the standard deviation of 
the activity usage count and Z is a semantic variable. 

All of the continuous usage count values were normalized to discrete preference scores (PS) and 0.3 is 
selected to cluster students into suitable groups by using Eq. (2) with Z < -0.3, -0.3 A Z A 0.3, and Z > 0.3, 
to respectively represent inactive, neutral and active preferences. The preference score (PS) is the degree of 
preference that a student demonstrates for an activity, which is defined as in Eq. (3). The preference score is 
1 if the usage count, X >x + 0.3 cj x , and the preference score is -1, if the usage count, X <x- 0.3 rr x . 0 
represents a neutral preference. 

( 1 , when X > x + 0.3cr x 

Preferece Score (PS) = I —1, when X < x — 0.3 ct x (3) 

v 0, otherwise 

The student-activity preference score matrix is shown in the following Table 1. 


Table 1 . Student-activity preference score matrix 


Student ID 

View 

courses 

Complete 

feedback 

Add post 
on forum 

Update post 
on forum 

View discussion 
on forum 

1 

-i 

0 

1 

0 

1 

2 

0 

-1 

0 

-1 

1 

3 

1 

0 

-1 

1 

0 

4 

0 

1 

0 

1 

-1 


4. EXPERIMENTAL RESULTS 
4.1 Experiment Setup and Dataset 

A dataset obtained from the learning management system (LMS) of a hospital was used for the experiment. 
The hospital is a medical center in northern Taiwan. This study was approved by the hospital ethics 
committee. Clinical teachers added learning resources (course, teaching material videos, or questionnaires) 
and activities (forum, chat room). Students viewed the courses, talked in chat rooms, discussed on forums 
and completed feedback on the LMS. The details are shown in Figure2. 

The experiment dataset was extracted from the learning management system of the hospital, for the first 
semester of 2013. There are 10,637 items of student activity transaction data on the LMS. The dependent 
variable, support and confidence thresholds for the association rules were set to 0.5, 0.2 and 0.6. These 
thresholds were set based on our observation on the online learning behaviors of the students. The most 
popular activities are viewing the course, completing feedback, adding and updating posts and viewing 
discussions on a forum. 
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Figure 2. Knowledge sharing (left) and a discussion on a forum (right) on the e-learning platform 


4.2 Student Cluster Identification 


It is firstly necessary to identify the characteristics of each student cluster, i.e. the activity preferences for 
each student cluster. The average usage count for activity X in a cluster is compared to the average usage 
count x plus/minus the standard deviation, o x , for all students, to obtain the cluster preference score 
(PS).using Eq. (2). The preference score for an activity in the cluster ID is PS(X^ c “^f t r y ID ). For example, 
PS(Xy iew course ) = —1 and PS(Xy iewcourse ) = 1. The activity preference scores for two clusters are shown 
in Table 2. The preference scores for activities such as viewing the course, completing feedback, adding 
posts, updating posts and viewing discussions on a forum in cluster 0 are PS( Xy iewcourse ) , 
PS(Xcomplete feedback)’ f^^Add post)’ f^(^Update post)’ PS(Xyj ew discussion), which is (-1, -1, -1, -1, -1) and 
the preference scores for these activities in cluster 1 are PS( Xy iew course) - PS( Xc omplete f eedback ) , 

PS(XAdd post)’ P^^Xupdate post)’ PS(Xy; ew discussion)’ which is (1, 1, 1, 1, 1). 


Table 2. Activity preference scores for students in the two clusters 


Cluster ID 

Activity 

Cluster 

All 

Preference 

X 

X 

X - 0.3 a x 

X + 0.3 o x 

Score (PS) 


course 

recent 

3.0 

3.0 

2.4 

3.6 

0 


view 

269.3 

399.0 

319.8 

478.2 

-1 


feedback 

complete 

7.0 

9.0 

7.8 

10.2 

-1 


folder 

view 

12.0 

10.0 

7.9 

12.1 

0 

0 


add discussion 

2.0 

2.0 

2.0 

2.0 

0 



add post 

38.3 

53.0 

44.3 

61.7 

-1 


forum 

delete post 

1.0 

1.0 

1.0 

1.0 

0 



update post 

1.0 

3.0 

2.4 

3.6 

-1 



view discussion 

294.3 

402.0 

332.7 

471.3 

-1 


course 

recent 

2.5 

3.0 

2.4 

3.6 

0 


view 

519.3 

399.0 

319.8 

478.2 

1 


feedback 

complete 

14.7 

9.0 

7.8 

10.2 

1 


folder 

view 

13.3 

10.0 

7.9 

12.1 

1 

1 


add discussion 

3.0 

2.0 

2.0 

2.0 

1 



add post 

78.5 

53.0 

44.3 

61.7 

1 


forum 

delete post 

1.5 

1.0 

1.0 

1.0 

1 



update post 

5.5 

3.0 

2.4 

3.6 

1 



view discussion 

531.3 

402.0 

332.7 

471.3 

1 
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The usage counts for these activities for cluster 1 are twice those for cluster 0. Cluster 0 is an inactive 
group and cluster 1 is an active group. Students in the inactive group (cluster 0) seldom view the course, 
complete feedback, add posts, update posts, or view discussions on a forum. However, the active group 
(cluster 1) students have the opposite learning behaviors. Table 2 shows that the usage counts for adding 
posts, updating posts and adding discussions for cluster 1 are all larger than those for cluster 0. Students in 
cluster 1 show confidence in self-expression. The usage counts for recent course, viewing folders and 
deleting posts for cluster 0 are all close to those for cluster 1 . “Recent course” means that students check the 
latest course status on the LMS. “Viewing folders” means that students download the necessary files, e.g., 
teaching plans and material, which are placed in folders on the LMS. “Deleting posts” means that students 
delete their post after adding a post on a forum. In summary, students in both groups download the necessary 
teaching material files, check the latest course information and delete their posts on a forum. 

4.3 Activity Association Rules for Student Clusters 

Table 3 shows the association rules between activities for cluster 0, i.e., from activity (X) to activity (Y). 
Students fill out the questionnaire, add discussions on a forum and then view discussions and search for the 
clinical questions on the platform. After adding a discussion on the forum, they view resources to find the 
answer to a clinical question. The resources are the teaching material files, which are provided by the clinical 
teacher. It is interesting that students in cluster 0 often remember to logout from the system and they pay 
more attention to the security of personal information than students in cluster 1. 

Table 3. Association rules for activities for cluster 0 


Cluster ID 

Activity (X) 


Activity (Y) 



recent 

-> 

user 

update 


course 



forum 

view forum 


view 


resource 

view 





user 

login, logout, update, view all 





forum 

view forum 


feedback 

complete 

> 

resource 

view 





user 

login, logout, update, view all 


folder 

view 


user 

logout, update 

0 




forum 

view forum 


add discussion 


resource 

view 





user 

login, logout, update, view all 





forum 

view forum 


forum 

add post 

> 

resource 

view 




user 

login, logout, update, view all 



delete post 


user 

logout, update 





forum 

view forum 



view discussion 


resource 

view 





user 

login, logout, update, view all 


Table 4 shows the association rules between activities for cluster 1, i.e., from activity (X) to activity (Y). 
Students in cluster 1 often view a uniform resource locator (URL) and resource after viewing the course and 
completing feedback. The URL is a web link to teaching videos on YouTube. Students in cluster 1 review the 
lesson on the platform after it is taught. In addition, students in cluster 1 keep in touch with teachers or 
classmates by writing messages. Students in cluster 1 also often surf many websites at the same time and are 
more often compulsorily logged out of the after 5 minutes of no activity. 

Table 3 and Table 4 show that students in both groups view other classmate’s recent profiles, which 
include information on the last login to the system. Students wish to know whether classmates view their 
recent posts and discussions on a forum, after they add posts and discussions on a forum. Students often 
check classmates’ latest login time to the system. They also often update their personal photos on the LMS. 
These are similar social behaviors to those seen on Facebook. It is found that students care about the peer 
status in the collaborative environment. Peer interaction is important during an internship. Students construct 
their own knowledge bases and learn from each other (Wilson and Stacey, 2004). In addition, students often 
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observe other students learning status and wish to attract their attention. When they add a new topic for 
discussion on a forum, they view the posts that other classmates add or update. At the same time, they 
regularly update their photos and personal information on their own home pages. They attract the attention of 
peers for personal and social reasons. 

Table 4. Association rules for activities for cluster 1 


Cluster ID 

Activity (X) 

-* 

Activity (Y) 


course 

recent 


user 

login, update 





message 

write 


course 

view 


resource 

view 



url 

view 





user 

login, update, view all 




> 

message 

write 


feedback 

complete 


resource 

view 

1 

> 

url 

view 





user 

login, update, view all 


folder 

view 

> 

user 

login, update, view all 



add discussion 

> 

user 

login, update, view all 



add post 

> 

user 

login, update, view all 


forum 

delete post 

-► 

user 

login, update 



update post 

-* 

user 

login, update, view all 



view discussion 

-> 

user 

login, update, view all 


5. CONCLUSION 

After cluster analysis, the average usage counts for viewing the course, completing feedback, adding posts, 
updating posts and viewing discussions on a forum for the active group are twice those for the inactive group. 
Therefore, students in the active group often view forums and resources after viewing the course, completing 
feedback, adding posts and viewing discussions on a forum. The student learning behavior patterns for each 
cluster can be derived, based on the association rules between activities for the cluster. Next year, when there 
are new interns, teachers can derive their clusters and association rules, to determine students’ behavior, and 
using their behaviors and the association rules, give proper guidance and resources, to allow students to be 
taught in accordance with their aptitude. However, this study has some limitations. Some students were not 
familiar with the e-learning platform. In the future, the e-learning platform will be introduced before the 
internship. 
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ABSTRACT 

The main focus of this study is to determine if systems development methodologies (SDMs) are being utilised in the 
development of electronic learning systems in South Africa. Electronic learning, or e-learning, is being employed to 
educate millions of learners, students and employees around the world and it is a critical component of modern 
educational systems. To ensure that e-learning systems of outstanding quality are being developed, it is therefore crucial 
that systems development methodologies are being used as they can have a significant impact on the development 
process. By utilising a survey as the main research method meaningful results were obtained. This study gave some 
insights into how learning management system procurement and development is being done in South Africa and revealed 
that the use of open-source systems currently exceeds the use of proprietary systems. The results of the research showed 
that systems development methodologies (e.g. Object-Oriented Analysis and Rapid Application Development) are being 
used in the development of e-learning systems. 


KEYWORDS 

e-learning, learning management systems, systems development methodologies. 


1. PROBLEM STATEMENT AND RESEARCH AIM 

Advances in information and communication technologies (ICT) have made huge volumes of information 
available to millions of people around the world. Some reports suggest that the U.S. workforce spends well in 
excess of $100 billion a year on job training (Clark & Meyer, 2008). 

Garrison (2011) believes it is of the utmost importance that higher educational institutions need to 
understand and embrace the increasing importance of technology in an educational environment. It is obvious 
that our way of learning will have to adapt to this inundation of information. 

According to Holmes and Gardner (2006) e-learning has properties that can overcome certain limitations 
of traditional learning, specifically limitations regarding the set times and locations for learning. 

As e-learning is becoming ever more popular, it has almost become synonymous with education. It is 
being used in universities for educational purposes, by corporations to train their staff, in primary and 
secondary schools to teach learners, etc. Almost all forms of training and education done nowadays, have an 
e-learning component. 

With the dramatic increase in the popularity and use of e-learning and e-learning systems it becomes a 
necessity to ensure that systems of a high quality are being developed. 

In another and not unrelated field of Systems Development Methodologies (SDMs), much more research 
has been done and the value of SDMs is proven and documented. However, there is still a lack of empirical 
evidence on the actual use and effectiveness of SDMs and even more so with its use and effectiveness in the 
development of e-learning systems. 

E-learning systems, being an emerging subset in software systems technology, could well profit from the 
benefits SDMs have to offer. 

The aim of this study was to research the use of SDMs in the development of e-learning systems in South 
Africa, specifically to determine if open-source learning management systems are gaining ground on their 
proprietary counterparts and also to determine relationships between the type of industry and the learning 
management platform. The study makes a contribution to the discipline of information systems, and more 
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specifically, learning management systems, by providing insights with regard to the use of SDMs in 
developing LMSs. This study has shed some light on what the South African e-learning market looks like. 
The researchers obtained a holistic picture of how e-learning systems (or more specifically, learning 
management systems) are being procured or developed in South Africa. 


2. LITERATURE STUDY: E-LEARNING 

This section gives a background on what is implied by the terms e-learning and e-learning systems and 
confers the difference between open-source and proprietary e-learning systems. 

2.1 Definition of e-learning 

Over the last decade or so, there has been some contention over the exact definition of e-learning and the 
terminology associated with e-learning. Some authors see e-learning as an overarching activity that involves 
any type of learning that is supported by ICT. This overarching term has been referred to as educational 
technology, communication and information technologies, technology-enhanced learning or web-based 
training. 

There are many disparate definitions of e-learning but for the purposes of this study e-learning can be 
seen as a medium for delivering and facilitating learning, through electronic means. 

2.2 E-learning Systems 

A software system helps manage all the different technologies, concepts and aspects pertaining to a specific 
problem. 

The Joint Information Systems Committee defines the term Virtual Learning Environment (VLE) as the 
components in which learners and tutors participate in on-line interactions of various kinds, including on-line 
learning (As quoted in Weller, 2007). VLEs can be described, in layman’s terms, as software systems that are 
specifically developed to facilitate teaching and learning in an educational environment. VLEs is mentioned 
as a precursor, for what we know today as Learning Management Systems (LMSs). Paulsen (2002) defines 
an LMS as a broad term that is used for a wide range of systems that organise and provide access to on-line 
learning services for students, teachers and administrators. The definition Weller (2007) provides, best 
explains an LMS as a software system that combines a number of different tools that are used to 
systematically deliver content on-line and to facilitate the learning experience around that content. 

For the purposes of this study, an LMS is a software system that is used to systematically deliver content 
on-line and to facilitate the learning experience around that content. 

When procuring an e-learning system one can divide the options into two main groups, namely: 
proprietary or open-source. Open-source software is getting increasingly popular due to the economical and 
modular benefits it can offer. There is a number of open-source LMS software available but the two that have 
the greatest market share are Sakai and Moodle. 

In contrast to open-source LMS software, proprietary software is produced and owned by a software 
company or software producer. Users of such software pay licensing fees and do not have access to the 
source code. Blackboard is one of the main competitors in the proprietary LMS market. 


3. LITERATURE STUDY: SYSTEMS DEVELOPMENT 
METHODOLOGIES 

The aim of this section is to give a review of systems development methodologies. Firstly, a brief overview 
on the background of SDMs will be given, then a clarification of the definition of SDMs will be made. 
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3.1 Background 

Prior to the 1970’s computer applications were implemented without the use of any formal systems 
development methodologies. The success or failure of such systems depended largely on the skill and 
experience of individual programmers. History shows us that systems development methodologies came into 
existence to address the shortcomings in existing techniques and to improve on the productivity and quality 
of software (Avison & Fitzgerald, 2002; Iivari et al., 2000). 

Huisman and Iivari (2006) argued that using a systems develop methodology is more effective than not 
using one. SDMs introduce a certain structure to the development process, which in turn improves the 
effectiveness of the design process and facilitates more consistent outcomes. 

Some of the better-known SDMs include Structured Systems Analysis and Design methodology 
(SSADM), Coad-Yourdon’s Object-Oriented Analysis Design methodology (OO ADM), Information 
Engineering (IE) and later on, Agile methodologies. 

3.2 Definition 

There has been much contention over the precise definition of the term. Systems Development Methodology. 
Wynekoop and Russo (1997) and Avison and Fitzgerald (2006) argue that there is no universally accepted 
and exact definition of what is implied by SDMs. This makes defining SDMs challenging. 

Some examples of the disparate definitions follow. A systems development methodology is a systematic 
procedure for completing either a system or one of several stages of the systems development life cycle. It 
consists of goals, principles and specific methods and tools, which are selected on the basis of an underlying 
rationale or system development philosophy (Iivari el al., 1999). Wynekoop and Russo (1995) defined a 
methodology as an orderly approach to carry out at least one stage of the systems development life -cycle, by 
using relevant tools, techniques, or guidelines, based on an underlying philosophy. Avison and Fitzgerald 
(2006) defined an SDM as a way to accomplish the development (or part of the development) of software, 
established on a set of rationales and an underlying philosophy. This includes a definition of phases, tasks, 
tools, guidelines and documentation. 

The following definition of a systems development methodology was developed (Huisman, 1999); 

• Systems development approach 

This can be defined as the philosophical view on which the methodology is built. Thus, the 
set of goals, guiding principles and beliefs, basic concepts and principles of the systems 
development process that drive interpretations and actions in systems development (Iivari et al., 
1998; Iivari et al., 1999). Examples of systems development approaches are the structured 
approach, object-oriented approach, information modelling, etc. 

• Systems development process model 

Wynekoop and Russo (1993) define a process model as a representation of the sequences of 
stages through which a system evolves. Some examples of process models are the linear life- 
cycle model, the spiral model and incremental model. 

• Systems development method 

A method is a systematic approach to conducting at least one complete phase of systems 
development, consisting of a set of guidelines, activities, techniques and tools, based on a 
particular philosophy of systems development and the target system (Wynekoop & Russo, 
1993). Examples include IE, SSADM, etc. 

• Systems development technique 

A systems development technique consists of a well-defined sequence of actions, ensuring 
successful results if used correctly (Iivari et al., 2000; Brinkkemper, 1996), for example entity 
relationship diagrams and data flow diagrams. 

This definition of SDMs implies that there is a development approach that guides the development. All 
the facets that need to be included in the development are underpinned in this approach. The process model, 
in turn, defines the order in which the development steps are carried out and is dependent on the development 
approach. The development method is what has to be done in order to develop the system, given the 
approach. The techniques are instruments used in accomplishing the steps of the development. All this adds 
up to what is called a systems development methodology. This encompassing definition will also be used for 
this study. 


199 



ISBN: 978-989-8704-08-5 © 2014 


4. DATA COLLECTION AND ANALYSIS 

In the field of information systems, surveys are a popular strategy to employ in the collection of empirical 
evidence. A questionnaire, as the measurement element, was developed in collaboration with a statistical 
consultation service. The constructs, identified from the conceptual research model, were operationalised by 
selecting measurement scale items (questions) and scale types. The questions were adapted from previous 
research studies, which proved to be reliable. The questionnaire was concise and relevant and contained 
mostly leading, importance and 5-point Likert scale questions. It was distributed electronically, as a macro- 
enabled Excel file, to personnel at institutions of higher education in South Africa that are responsible for 
developing and/or deploying e-learning systems. Software companies in South Africa, which develop e- 
learning solutions, were also targeted. The questionnaire had extensive built-in entry validation to ensure that 
the respondents fill in the correct values as well as coding to assist in data analysis. 

Fifty responses were received from a possible one hundred and twenty-five responses. Therefore, the 
participation rate equalled 40% with 50 cases available for data analysis. 


5. RESULTS 

This section gives a summary of the background information of all the respondents that completed the 
survey. 

Industry the respondents work in 

Just fewer than 60% of the respondents stated that the type of industry they work in can be described as 
“Academic”. The rest (40.43%) were in the “Private Sector”. 

Number of learners, students or employees 

Almost 58% of the LMSs are being used to train or educate 10 000 or more people. 

LMS Platform 

Almost 60% of respondents use an open-source LMS environment and not proprietary. 

Procurement method of LMS 

Respondents were asked whether they purchased, developed in-house, outsourced, use an open-source 
(as-is) or adapted from an open-source LMS. Just fewer than 32% of the respondents indicated that they 
adapted their current LMS from an open-source based LMS. See Table 1. 


Table 1. Procurement method of LMS 


Procurement method 

Percentage 

Purchased 

23.40% 

Developed in-house 

17.02% 

Outsourced 

2.13% 

Open-source (used as-is), specify which: 

25.53% 

Adapted from open-source system, specify which: 

31.91% 


Perceived success of the LMS 

Respondents were asked to what extent they agree on statements regarding their current LMS. An 
overwhelming 84 . 91 % > of the respondents answered in the affirmative, by marking either “Agree” or 
“Totally Agree”, on the different options of this question. See Table 2. 


1 This percentage was calculated by adding the Agree and Totally Agree columns and averaging it for all the variables across the 
complete question. 
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Table 2. Perceived success of LMS 


Statement 

Totally 

Disagree 

Disagree 

Percentage 

Neutral 

Agree 

Totally 

Agree 

The system is functional 

0.00% 

0.00% 

4.26% 

53.19% 

42.55% 

The system is reliable 

0.00% 

6.38% 

6.38% 

42.55% 

44.68% 

The system is maintainable 

0.00% 

4.26% 

4.26% 

42.55% 

48.94% 

The system is portable 

0.00% 

2.13% 

25.53% 

42.55% 

29.79% 

The system is efficient 

0.00% 

6.38% 

8.51% 

42.55% 

42.55% 

The system is usable 

0.00% 

0.00% 

8.51% 

42.55% 

48.94% 

The developed system meets user needs 

2.13% 

2.13% 

10.64% 

53.19% 

31.91% 

The documentation of the developed system is good 

6.38% 

14.89% 

12.77% 

36.17% 

29.79% 

Overall, the developed system is of high quality 

2.13% 

2.13% 

12.77% 

38.30% 

44.68% 

Overall, the users are satisfied with the developed system 

0.00% 

2.13% 

12.77% 

53.19% 

31.91% 

Overall, the developed system is a success 

0.00% 

4.26% 

4.26% 

55.32% 

36.17% 


Satisfaction with the LMS platform 

To statements regarding the e-learning platform, 44.68% of the respondents agreed that they were 
satisfied with their current platform and 29.79% totally agree to being satisfied with their platform. Almost 
62% of the respondents would consider using an open-source LMS environment for future projects and 
46.81% “Disagree” or “Totally Disagree” about considering in-house development for future LMS projects. 
See Table 3. 


Table 3. LMS platform satisfaction and procurement for future LMS projects 


Statement 

Totally 

Disagree 

Disagree 

Percentage 

Neutral 

Agree 

Totally 

Agree 

I'm totally satisfied with our current platform 

4.26% 

6.38% 

14.89% 

44.68% 

29.79% 

I would consider using an open-source e-learning system 
for future projects 

6.38% 

12.77% 

19.15% 

27.66% 

34.04% 

I would consider purchasing a proprietary e-learning system 
for future projects 

12.77% 

23.40% 

29.79% 

23.40% 

10.64% 

I would consider the in-house development of an e-leaming 
system for future projects 

27.66% 

19.15% 

19.15% 

17.02% 

17.02% 


The use of formal SDMs 

Respondents were asked to indicate whether they used formal systems development methodologies to aid 
in systems development. Almost 80% indicated that they use formal SDMs in development. 

To what extent were standard SDMs used 

On the question that asked respondents to indicate which SDMs they use and to what extent, Object- 
Oriented Analysis (OOA) and Rapid Application Development (RAD) were most frequently selected, with 
47.06% and 38.24% of the respondents respectively, using it to a large extent. See Table 4. 


Table 4. SDMs used in LMS development 


SDM Used 

Not At All 

Percentage 

To Small To Large 

Extent Extent 

To Full 
Extent 

STRADIS (Structured Analysis, Design & Implementation of 
Information Systems) 

47.06% 

23.53% 

23.53% 

5.88% 

OOA (Object-oriented Analysis) 

14.71% 

35.29% 

47.06% 

2.94% 

RUP (Rational Unified Process) 

70.59% 

23.53% 

5.88% 

0.00% 

XP (Extreme Programming) 

52.94% 

29.41% 

14.71% 

2.94% 

RAD (Rapid Application Development) 

35.29% 

23.53% 

38.24% 

2.94% 

ETHICS (Effective Technical & Human Implementation of 
Computer-based Systems) 

85.29% 

8.82% 

2.94% 

2.94% 

IE (Information Engineering) 

47.06% 

38.24% 

11.76% 

2.94% 

SSM (Soft Systems Methodology) 

76.47% 

17.65% 

5.88% 

0.00% 

Other, specify: 

76.47% 

2.94% 

17.65% 

2.94% 


201 



ISBN: 978-989-8704-08-5 © 2014 


Stringent use of SDMs 

Almost 60% of the respondents indicated that they adapted the SDM they used, depending on the specific 
project requirements. 

Performance expectancy and perceived support of the SDM 

The respondents were generally positive on the use, functionality and benefits of SDMs - as can be seen 
in table 5. 


Table 5. Performance expectancy of the SDM and perceived SDM support 


Statement 

Totally 

Disagree 

Disagree 

Percentage 

Neutral 

Agree 

Totally 

Agree 

Using an SDM is recommended in my work 

0.00% 

0.00% 

20.59% 

58.82% 

20.59% 

The result of using an SDM is clear to me 

2.94% 

2.94% 

20.59% 

41.18% 

32.35% 

I would be able to communicate to others the consequences 
of using an SDM 

2.94% 

11.76% 

17.65% 

55.88% 

11.76% 

I would be able to explain why using an SDM may or may 
not be beneficial 

8.82% 

0.00% 

20.59% 

52.94% 

17.65% 

The application of the SDM is clear to me 

8.82% 

8.82% 

14.71% 

52.94% 

14.71% 

The benefits of using the SDM is apparent 

5.88% 

2.94% 

5.88% 

58.82% 

26.47% 

Overall, I believe the SDM is user-friendly and easy to 
apply 

2.94% 

5.88% 

26.47% 

61.76% 

2.94% 

rrJ 

The SDM enable me to complete tasks quicker 

2.94% 

2.94% 

35.29% 

44.12% 

14.71% 

The quality of my work is improved by using the SDM 

2.94% 

0.00% 

20.59% 

52.94% 

23.53% 

It is easier to accomplish my work by using the SDM 

2.94% 

2.94% 

29.41% 

47.06% 

17.65% 

The effectiveness of my work in systems development was 
enhanced using an SDM 

2.94% 

2.94% 

41.18% 

38.24% 

14.71% 

I had greater control over my development work when 
using an SDM 

2.94% 

0.00% 

20.59% 

61.76% 

14.71% 

The SDM is compatible with my development work 

2.94% 

2.94% 

26.47% 

58.82% 

8.82% 

The SDM fits well in the way I like to work 

5.88% 

0.00% 

26.47% 

52.94% 

14.71% 

I was permitted to use the SDM on a trial basis to see what 
it can do 

1 1 .76% 

5.88% 

26.47% 

50.00% 

5.88% 


Perceived impact of the SDM on the LMS 

The respondents were asked to indicate whether they agree with statements regarding the effect SDMs 
had on their e-learning system in terms of functionality, reliability, maintainability, efficiency, quality, 
usability and user satisfaction. Notably, about 56% of the respondents agreed and about 12% totally agreed 
that the e-learning system was more reliable as result of using an SDM in development. On average, 58% 2 of 
the respondents indicated that they “Agree” or “Totally Agree” that their developed e-learning system was 
more functional, reliable, maintainable, efficient, of better quality, more usable and that users are more 
satisfied with the e-learning system, as result of using SDMs to develop the system. See Table 6. 


Table 6. Impact of SDM on LMS 


Statement 

Totally 

Disagree 

Disagree 

Percentage 

Neutral 

Agree 

Totally 

Agree 

The e-learning system is more functional as result of 
using the SDM 

0.00% 

0.00% 

44.12% 

41.18% 

14.71% 

The e-learning system is more reliable as result of using 
the SDM 

0.00% 

0.00% 

32.35% 

55.88% 

11.76% 

The e-learning system is more maintainable as result of 
using the SDM 

0.00% 

0.00% 

32.35% 

41.18% 

26.47% 

The e-learning system is more efficient as result of using 
the SDM 

0.00% 

0.00% 

47.06% 

38.24% 

14.71% 


2 This percentage was calculated by adding the Agree and Totally Agree columns and averaging it for all the variables across the 
complete question. 
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The e-learning system is of better quality as result of 
using the SDM 

0.00% 

2.94% 

35.29% 

50.00% 

11.76% 

The e-learning system is more useable as result of using 
the SDM 

0.00% 

0.00% 

47.06% 

35.29% 

17.65% 

The users of the e-leaming system is more satisfied as 
result of using the SDM 

0.00% 

2.94% 

50.00% 

35.29% 

11.76% 

Reasons for not using SDMs 






The next question tried to ascertain the reasons 

for not using 

; SDMs. 

The results can be seen in table 7. 

More than 60% of the respondents confirmed (Agree and Totally Agree) that there 

was a lack of staff 

experienced in implementing SDMs in their development work. 





Table 7. Reasons for the non-use of SDMs 







Percentage 



Statement 

Totally 

Disagree 

Disagree 

Neutral 

Agree 

Totally 

Agree 

My organisation/IS department doesn't require the use 
of SDMs 

26.47% 

50.00% 

0.00% 

14.71% 

8.82% 

SDMs are complex or difficult to use 

14.71% 

55.88% 

23.53% 

5.88% 

0.00% 

Our current processes for development are adequate 

2.94% 

29.41% 

23.53% 

44.12% 

0.00% 

The benefits of using SDMs are not extensive enough 

8.82% 

64.71% 

23.53% 

2.94% 

0.00% 

The initial cost of procuring an SDM are high 

14.71% 

41.18% 

32.35% 

11.76% 

0.00% 

There is a lack of staff experienced in SDMs to 
implement such a strategy 

5.88% 

23.53% 

8.82% 

58.82% 

2.94% 

There is a lack of management support for the use of 
SDMs 

5.88% 

14.71% 

41.18% 

23.53% 

14.71% 


The need for an LMS-specific SDM 

With the last question of the questionnaire the researchers wanted to ascertain, by means of certain 
statements, if there was room for a newly developed SDM designed specifically for e-learning systems. 
Almost 75% of the respondents agreed that there was room for a newly designed SDM for developing LMSs. 


6. CONCLUSION 

The descriptive statistics that are presented in this paper reveal some interesting facts about learning 
management systems in South Africa and the use of systems development methodologies when developing 
LMSs. 

Both academic institutions and the private sector were approached to participate in this study. The 
respondents that returned the questionnaire were mostly developers, project leaders and people involved in 
the deployment of LMSs or involved with user support. More than half of the respondents indicated that their 
LMSs are being used for 10 000 or more learners and that they use an open-source LMS. Almost a third of all 
the respondents adapt their open-source LMS to suit their needs. Overall, the respondents perceived their 
LMS to be successful with almost half of them indicating that they agreed completely that their LMSs are 
efficient. A great number of respondents answered in the affirmative that they would consider an open-source 
LMS for future projects. 

Almost 80% of the respondents indicated that they made use of formal SDMs. This is in line with recent 
studies done in South Africa. It appears that development teams involved with LMS projects are relatively 
small with more than half of the respondents indicating project teams of five members or less. Many of the 
standard SDMs are not being used to their full extent but the two standout SDMs were Object-Oriented 
Analysis and Rapid Application Development, which is an agile methodology. Almost none of the 
respondents use an SDM rigorously and more than half adapt them based on the specific needs of the project. 
This is in line with what is known about the contingent use of SDMs. 
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More than three quarters of the respondents were positive on the support and benefits that SDMs provide 
and almost as many answered in the affirmative that SDMs could have a positive impact on the development 
process. Many of the respondents were positive that SDMs could be advantageous in the development of 
LMSs in terms of the reliability, maintainability, efficiency and the quality of the LMS. Almost three quarters 
of the respondents indicated that they would adapt SDMs for future projects as needed for the specific 
project. Respondents indicated a substantial lack of personnel experienced in the use of SDMs as the major 
reason for the non-use of SDMs in development projects. They agreed on statements regarding the benefits 
SDMs have to offer as well as SDMs being recommended in their development work. 

Almost three quarters of the respondents felt that there may be room for a newly designed SDM that 
could enhance their LMS development work. 

This study may serve as a stimulus for future research in the field of learning management systems and 
more specifically the development of LMSs by using systems development methodologies to enhance the 
chances of success for those systems. A holistic picture was drawn on what the South African e-learning 
market looks like and it was determined that the extent of use of open-source LMSs exceeds what was 
initially believed. It was substantiated that SDMs are being used in the development of LMSs. The study also 
unlocks various future research possibilities in the emerging field of LMS development. 


REFERENCES 

Avison, D.E. & Fitzgerald, G. 2002. Information systems development: methodologies, techniques and tools. 3rd ed. 
London: McGraw-Hill. 608 p. 

Avison, D.E. & Fitzgerald, G. 2006. Information systems development: methodologies, techniques and tools. 4th ed. 
Maidenhead: McGraw-Hill. 645 p. 

Brinkkemper, S. 1996. Method engineering: engineering of information systems development methods and tools. 
Information and Software Technology , 38:275-280. 

Clark, R.C. & Mayer, R.E. 2008. E-learning and the science of instruction: proven guidelines for consumers and 
designers of multimedia learning. 3rd ed. San Francisco: Pfeiffer. 528 p. 

Garrison, D. R. 2011. E-learning in the 21st century: A framework for research and practice. 2nd ed. London: Taylor & 
Francis. 161 p. 

Holmes, B. & Gardner, J. 2006. E-learning: Concepts and practice. London: Sage Publications. 200 p. 

Huisman, H.M. 1999. The deployment of systems development methodologies: A South African Experience, Thesis - 
PhD. 

Huisman, H.M. & Iivari, J. 2006. Deployment of systems development methodologies: perceptual congruence between 
IS managers and system developers. Information & Management, 43(1 ):29-49. 

Iivari, J., Hirscheim, R. & Klein, H.K. 1998. A paradigmatic analysis contrasting information systems development 
approaches and methodologies. Information Systems Research, 9(2): 164-193. 

Iivari, J., Hirschheim, R. & Klein, H.K. 1999. Beyond methodologies: Keeping up with information systems 
development approaches and methodologies. Information Systems Research, 9(2): 1 64- 193 . 

Iivari, J., Hirschheim, R. & Klein, H.K. 2000. A dynamic framework for classifying information systems development 
methodologies and approaches. Journal of Management Information Systems, 17(3): 179-218. 

Paulsen, M.T. 2002. Online education systems: discussion and definition of terms. NK1 Distance Education. Jul. 2002. 

Weller. M. 2007. Virtual learning environments: using, choosing and developing your VLE. New York: Routledge. 192 
P- 

Wynekoop, J.L. & Russo, N.L. 1993. Studying systems development methodologies: Unanswered questions and the 
research gap. (In DeGross, J.I. et al., eds. Proceedings of the Fourteenth International Conference on Information 
Systems, Orlando, FL, p. 181-190). 

Wynekoop, J.L. & Russo, N.L. 1995. Systems development methodologies: unanswered questions. Journal of 
Information Technology. 10(2):65-73. 

Wynekoop, J.L. & Russo, N.L. 1997. Studying systems development methodologies: an examination of research 
methods. Information Systems Journal, 7:47-65. 


204 



International Conference e-Leaming 2014 


ASSESSMENT OF THE USE OF ONLINE COMUNITIES TO 
INTEGRATE EDUCATIONAL PROCESSES 
DEVELOPMENT TEAMS: AN EXPERIENCE IN POPULAR 
HEALTH EDUCATION IN BRAZIL 


Elomar Castilho Barilli 1 , Stenio de Freitas Barretto 2 , 

Carla Moura Lima 3 and Marco Antonio Menezes 4 

1 Full Researcher in Public Health - Distance Education Coordination, Sergio Arouca National Public Health School, 
Oswaldo Cruz Foundation. Rua Leopoldo Bulhoes, 1480, sala 301 - Manguinhos - Rio de Janeiro - 
2 Online Connections Center of the Popular Health Education Qualification Program - EdPopSUS 
3 Technical Coordination of the Popular Education Qualification Program 
4 Coordinator of the Popular Education Qualification Program 


ABSTRACT 

This paper is intended to share the results of the assessment of the use of the Online Work Community (OWC), 
developed in the Moodle technology that was used as an instrument to facilitate the educational and operational 
processes, intended to share problems and proposals for solution among the 470 members of the development teams, 
made up of educators, experts, technicians, managers and supporters, connected to the Popular Health Education 
Qualification Program, aimed at the qualification of the Basic Healthcare (Atengao Bdsica a Saikle - EdPop-SUS), 
through the examination of the issues concerning educational practices related to health in the disadvantaged 
communities of the Federal District and 8 other Brazilian states. The Program, based on critical pedagogy (FREIRE, 
1997), sets forth three stages of a familiarization course aimed at 17 thousand healthcare professionals, and another one 
for a deeper level of understanding, for 8 thousand alumni of the first course. The assessment methodology is based on 
the Dialogue Assessment (ROMAO, 2005), in which the assessment took two paths for analysis, a quantitative one and a 
qualitative one. The first one occurred by means of a survey of the perceptions of the participants, recorded in two online 
self-applied questionnaires, with open and closed questions that focused on the access, guidance/navigation and 
functionalities. The other one through the observation of the posts and interventions in the different online spaces of the 
OWC. The quantitative analysis ratified attendance as a still important characteristic in the Popular Education field, due 
to the difficulties pointed out by the participants regarding access (31%) and navigation and guidance (42%). The 
qualitative analysis, having as Speech Analysis as basis, revealed the wealth of the construction process by means of the 
shared statements and productions, totaling 159 products connected to culture and art. In addition, it evidenced the 
problems faced, the “political” category being the most significant one for the participants. Thus, the assessment 
concluded that the OWC allowed social integration between educators and team members, contributing to the 
implementation of the National Popular Health Education Policy, a government action that brings care, absence of 
centeredness. dialogue and respect to the community cultural diversity as premises for its implementation. 


KEYWORDS 

Popular Health Education, Online Communities, Community Healthcare Agents, Unified Healthcare System ( Sistema 
Unico de Saude - SUS). 
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1. INTRODUCTION 

Professional education currently works with elements connected to the civilization milestone of modernity 
that encompasses reflection, culture and subjectiveness. Within such context, the starting point for learning 
is the understanding of education as a social interaction experience by language and action (Vygotsky, 2003), 
intended for the creation of a learning, speech and practice community, in order to produce results, 
understanding for critical action, exercising learning through cooperation and autonomy, ensuring the 
centrality of the individual in the construction of knowledge and allowing cognitive and affective results, and 
those aimed at actions (COELHO, 1999). 

Regardless of whether the modality of education is class-attendance or at a distance, the perpetuity of 
such integration is sought, by means of Technologies considered to be mediators of human wealth; 
environments able to promote the meeting of people by transcending time and space; the so-called Online 
Environments. Beyond communication, within an educational context, such environments are expected to 
contribute to the intuitive search and shared construction of knowledge that is no longer individual, to gain a 
collective dimension. Thus, even if they are in different physical spaces, the social relationships gain power 
insofar as, in such environments, active expression and participation are valued. 

In relation to the concept of health, it currently transcends the purely biomedical understanding of 
absence of illness, to give preference to quality of life, incorporating the notion of promotion of health. This 
new paradigm presupposes social participation as a crucial element to reach equity, universality of assistance 
and integrality of healthcare, principles that are the goals of the Brazilian healthcare system - the Unified 
Healthcare System ( Sistema Unico de Saude - SUS). 

The structuring of public policies aimed at healthcare in communities goes through popular education 
strategies involving the Basic Healthcare professionals and the population itself, so that they see themselves 
as health promoters and, in case of the professionals, as healthcare agents in charge of the mediations in the 
health scope, within a concept of State. Logically, social actions related to health assume an educational axis, 
for it is only through it that new authority paradigms can be turned into new ways of seeing, understanding 
and working with health. Such new participative concept of healthcare may be appropriate in a country of 
continental dimensions. 

Popular Health Education (EPS) allows the approximation and dialogue among the popular knowledge, 
the scientific medical knowledge, the healthcare professionals and institutions. The EPS constitutes a 
movement expressed by the healthcare practices, production of shared knowledge and constitution of subjects 
and political players in the health field (BONETTI, PEDROSA, SIQUEIRA, 201 1). 

The main professionals connected to the EPS are the Community Healthcare Agent (Agente Comunitario 
de Saude -ACS) and the Health Surveillance Agent (Agente de Vigilancia de Endemias - AVE), people who 
were born, live and work in the communities, understanding their realities and healthcare needs. Such work, 
however, has been faced with outdated and authoritarian guidelines that set forth a mere fight against disease 
instead of working on its causes, together with the populations, incorporating the local realities involving 
cultural and environment dimensions. 

In October, 2011 the National Basic Healthcare Policy (PNAB/2011) stressed that Basic Healthcare 
occurs by means of care practices and democratic and participative management, also ratifying the 
educational activities as duties of such agents, so as to promote health and prevent disease. Two years later. 
Ordinance 2,761/2013 instituted the Popular Healthcare Education Policy (Polltica Nacional de Educa?ao 
Popular em Saude - PNEPS-SUS) that, among other elements, proposes a political and educational practice 
to guide the actions aimed at the [...] promotion, protection and recovery of health, based on the dialogue 
among the several kinds of knowledge, valuing popular knowledge, ancestrality and incentive to individual 
and collective production of knowledge...” (BRAZIL, 2013). 

As the main strategy to contribute to the consolidation of PNEPS-SUS, in October, 2013 the Popular 
Healthcare Education Qualification Program (EdPop-SUS) was launched, a fruit of the partnership between 
the Strategic and Participative Management Department of the Brazilian Ministry of Health, the National 
Public Healthcare School (Ensp) and the Joaquim Venancio Polytechnic Health School (EPSJV), the latter 
units being members of the Oswaldo Cruz Foundation (Fiocruz), a federal institution connected to the 
Ministry of Health, headquartered in the city of Rio de Janeiro and with 6 other regional centers in Brazil. 

EdPop-SUS was developed to qualify the educational practices in the basic healthcare field by means of 
the formation of ACS and AVEs that, mentioned in the Brazilian Healthcare System principles, develop 
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educational practices of social mobilization, health promotion and equity, its political and educational 
reference being the Popular Health Education (invitation to bid, 2014). It is structured in an awareness course 
made up of three 53-hour stages for 17 thousand agents and another stage, for a more in-depth understanding, 
for 8 thousand agents, qualified of the first offer. The EdPop-SUS is a great challenge in the Brazilian public 
health field, seeing as in addition to the social inclusion of such players, it also sets forth the digital inclusion 
of 17 thousand agents that have been historically deprived of access to consumer goods, including digital 
access technologies. 

Due to the fact that it involves the Federal District, in addition to 8 other Brazilian states, the Popular 
Health Education Qualification Program relies on the decentralized participation of a multi -professional staff 
of approximately 470 professionals, each of them dealing with the regional specificities typical of a 
continental country. Due to that, in addition to the educational dimension inherent to all collective 
construction process, such action has, as the crucial element of the shared management of processes, an 
Online Community containing spaces intended for the sharing of ideas and experiences, the Online Work 
Community - OWC. The interaction among the teams occurred by means of discussion forums in which the 
work teams exchanged experiences, impressions and doubts about the development and implementation of 
the course. 

This paper is intended to show the results of the assessment made with the faculty and work teams about 
the experience with the OWC, connected to the Popular Health Education Program 
(www6.ensp.fiocruz.br/edpopsus) in progress, pertaining to the first stage of the awareness course. 


2. THE ONLINE WORK COMMUNITY (OWC) ASSESSMENT AND ITS 
ANALYSIS 

The political and educational guideline of the Popular Health Education Awareness Course, based on socio- 
constructivist grounds, considers that learning is a complex process that occurs within a social context by 
means of the interaction among cultural agents that go through new experiences, which are compared with 
other experiences already lived, thud favoring the development of new mental schemes expressed in 
individual and collective knowledge (BARILLI and PESSOA, 2013). Add to that the critical pedagogy 
guideline that understands education as being the practice of freedom that, by means of the critical 
understanding of reality, renders the subject politically able to free him or herself from forms of oppression 
(SANTIAGO, 2012), considering that knowledge belongs to a greater act, that of knowing, thus being the 
epistemological element of the action of learning (FREIRE, 1997). 

Due to the innovative nature of the EdPop-SUS, the development and implementation activities were 
considered to be learning experiences for the 470 members of the teams. For such, in addition to the OLC, an 
Online Work Community (OWC) as made avialable, intended for the follow-up, from a management and 
interaction and support opportunity viewpoint, but The OLC is not the purpose of this paper, which is 
dedicated to the assessment of the Online Work Community (OWC). 

The OWC is an online environment supported by the Moodle technology, developed so as to promote 
integration between the work teams connected to the EdPopSUS and to facilitate the decentralized and shared 
management of the teaching and operational processes. 

We must stress the importance of being able to provide an environment for human 
communication/interaction, research and collective construction. 

The OWC’s structure expresses the work processes related to the development and implementation of the 
course, composed by: a) Online Mediators’ Room - a space where the mediators receive educational support 
from professional education experts, in addition to popular educators and learning instructors; b) 
Coordinators and teams forum - intended for the sharing of solutions and strategies implemented in the state 
centers; c) Exchange forum - This forum is for general use, intended for the sharing of News and 
documents, needs, doubts regarding the work process and; d) Doubts regarding the Online Work Community 
- support to the doubts regarding the navigation and use of the OWC’s tools. The support, in this space, is 
distributed by each participant state. 
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In addition to the process development teams, the Program also relies on the participation of the faculty 
(185 participants), which comprises three different profiles: 1) The Mediator - teachers responsible for the 
learning relationship with the student; 2) Popular Educator - professionals of any level of qualification, but 
who have solid experience in popular healthcare education practices with disadvantaged communities. Their 
duty is to provide support to the mediator in the teaching work, by means of their experience and popular 
educators; and 3) Learning Instructor - working as theoretical, methodological and educational reference for 
the faculty (mediators and popular educators). 

In order to be coherent with the educational guidelines adopted, the assessment of the OWC followed the 
“Dialogue Assessment” approach, that affirm that the purpose of which is to allow the different players 
involved, whose fields of judgment are often different, to be apt to take a stance in order to build, 
individually or collectively, a judgment that could be translated into action. Thus, having previously 
established criteria and methods as basis, the assessment subsidies the decision-making process that will 
support future actions (ROMAO, 2005). 

Within such logic, from a methodological point of view, the assessment followed a quantitative and 
qualitative guidance, the latter supported by the methods of content analysis and discourse analysis. Thus, the 
OWC assessment followed two paths, one with a quantitative emphasis and another with a qualitative one. 
The first one was through two online self-applied questionnaires, with closed questions, developed in the 
Online Assessment System , made available at the institutional web address (pesquisa.ead.fiocruz.br), touching 
upon the easiness of access, guidance/navigation, tools and functionalities. Due to its not being required, the 
first one, intended for the faculty, was replied to by only 127 people. The second questionnaire, aimed at the 
other members of the work teams, was replied by 43 people. 

The quantitative analysis was based on the results of the questionnaires, number of interactions and 
products shared in the different online spaces of the OWC. 

The second methodological path, of a qualitative nature, occurred by means of the observation of the 
“speech” of the participants, expressed in the posts and interventions in the different online spaces of the 
OWC. The assessment was therefore focused on the content of the interventions, seeking to group them into 
the following significant categories: experiences lived, problems faced, cultural expressions. 

The qualitative method of assessment adopted was the Speech Analysis, following the principle set forth 
by Rocha and Deusdara (2005), that the text of the informants’ replies forms a surface to be explored, from 
which "psychological qualities" emerged in relation to the object (p.33 1), herein considered to be the 
dialogues posted on the OWC. 


3. RESULTS 

The access to, as well as the guidance and navigation in the OWC, were considered difficult, perhaps due to 
the participants’ profiles, since in the Popular Education attendance is one of the main characteristics 
(Figures 1 and 2), followed by the difficulty in accessing Internet in some of the communities located in the 
countryside of Brazil, where the mediators live and work. 



Figure 1. Access to the OWC 


Figure 2. Guidance and navigation in the OWC 
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Even though considered to be simple activities, demands regarding difficulty in changing passwords, 
posting of images and sending of messages were recurrent. 

In spite of the difficulties evidenced by the results show, we must highlight the fact that in this first stage 
of the Course it was not possible to hold the (practical) training on the OWC, due to the unavailability of 
places with computers upon the initial qualification of the faculty, which could explain the participants’ 
difficulty in using the functionalities of their virtual spaces. Even so, table 1, shows the wealth of productions 
shared in the OWC. 


Table 1 . Survey of productions shares on the OWC, distributed by participant 


Products 

CE 

PE 

BA 

DF 

SE 

RJ 

SP 

RS 

PI 

TOTAL 

Poetry 

2 


6 

3 



2 


1 

14 

Pictures 

33 


22 

1 

4 

1 

21 



82 

Videos 







6 



6 

Texts 

3 



1 






4 

Assessments 

1 

2 

4 

1 

1 

1 

43 



53 

TOTAL 

39 

2 

32 

6 

5 

2 

72 

0 

1 

159 


States of Brazil: CE = Ceara. PE = Pernambuco, BA= Bahia, DF= Federal District, SE=Sergipe, 
RJ=Rio de Janeiro. SP=Sao Paulo, RS=Rio Grande do Sul, PI=Piauf. 


The qualitative analysis sought to prioritize the expressions of experiences lived, perceptions, doubts and 
confessions, and wow it contributed to the development and success of the Program and personal growth of 
the participants. 

Even though the use of all spaces had been instructed, there was a certain confusion in the posting. We 
can mention as an example the news forum, which was used to share reports on experiences lived during the 
course, in addition to requests for support regarding the procedures and use of the OWC. 

We noted that Learning Instructors (Lis), Popular Educators (EPs) and Mediators prioritized the posting 
of information on what was done (experiences) in the class-attendance moments, field work and impressions, 
as a way of sharing their experiences, which fact ratifies the hands-on learning of popular education. One of 
the most interesting contributions reported in the OWC was the mediators’ efforts to create electronic 
addresses (e-mail) for the students, in order to allow them to access the OLC. The expectations and concerns 
related to this activity were often shared in the OWC, in the Mediators’ Room space. 

For the analysis of the speech of the subjects, in the OWC online spaces, also according to Rocha and 
Deusdara (2005), the researcher makes a floating reading , which is the basis for him or her to formulate his 
or her hypotheses, to be validated or not in the subsequent stages. After that, the researcher extracts criteria to 
classify the results obtained into categories of signification (p.3 1 3). On the other hand, for the same writers 
the Speech Analysis Works on the meaning, and not only the content, seeking to bridge the distance between 
the research question and the question aimed at the participant, integrating textual organization (contents) and 
social context. The text is, thus, seen as the materialization of the speech (p.321). 

Thus, the analysis tool 157 posts in the OWC interaction spaces into account, of which the floating 
Reading as made, based on which 3 dimensions were formulated: experiences lived (97 posts), problems 
faced (more recurrent, with 112 posts) and cultural expressions (52 posts). Based on the dimensions 
surveyed, signification categories emerged, shown in Table 2. The “Political’’ category proved to be the most 
significant one for the participants, as shown by Graphic 1 . 


209 



ISBN: 978-989-8704-08-5 © 2014 


Table 2. Signification categories identified based on the posts to the OWC 


DIMENSIONS 

CATEGORIES 

POSTS AT THE OWC 

EXPERIENCES 

LIVED 

POLITICAL 

Let’s not be afraid to create... let’s dare! Going over the institutional walls doesn’t mean 
only opening the range of multiple possibilities of creation of knowledge. 

SOCIAL 

As a learning subject... I acquire voice and speech... here among you... I am concrete 
existence. . . turned into living contents. 

EDUCATIIONAL 

The search for acknowledgment of the forms of resistance of the community, as well as 
potentials, organization and popular mobilization, considering the development of a 
critical and reflective thinking on the concrete reality lived by the healthcare 
professionals in their daily work lives and their territory, connecting popular education to 
life. 

MANAGEMENT 

Knowledge of reality - knowing a bit of the local healthcare network, the history of the 
neighborhoods, of the people who live there, respecting the colleague, exchanging 
experiences, respecting the popular knowledge, listening to the community. 


PROBLEMS 

FACED 

POLITICAL 

The public policies confine the healthcare work with the communities to epidemiological 
surveys (filling out of forms). The data is more importante than people. 

SOCIAL 

Inability to handle social problems affecting the community’s healthcare levels: violence, 
alcoholism, drugs. 

EDUCATIONAL 

Adaptation of the teaching models to the Popular Education reality 

MANAGEMENT 

Social inclusion also comprises digital inclusion. 


CULTURAL 

EXPRESSIONS 

POLITICAL 

For the implementation of a policy with the PENEPS-SUS... ’’change is necessary”, for 
the forms of healthcare education... there must be “new reflections” and for the 
professionals and the community to realize that. 

SOCIAL 

Cultural expression as an element of community integration (the vaquejada holiday, 
bumba meu boi, rhyming, verse, poetry and cordel. 

EDUCATIONAL 

Cultural expression in the healthcare educational practices - speaking the “language” of 
the population in the communities. 

MANAGEMENT 

The expressions, discoveries of potential, a new look of the community. 



Experiences lived 
Problems faced 
Cultural expressions 


Political Social Educative Management 


Graph 1 . Posts on to the OWC pertaining to the signification categories 
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4. CONCLUSIONS 

One of the challenges faced by the PNEPS-SUS is the social insertion of the Brazilian disadvantaged 
communities. The path taken was the improvement of the basic healthcare levels through the qualification of 
the educational practices in the health segment, carried out by the healthcare agents that work in such 
communities. The barriers of access to production goods for the majority of the Brazilian communities has 
historically confirmed, in Popular Education, class attendance in the courses as a path for exchange and fight, 
materialized as popular movements that prioritize art and culture, a context that can explain the wealth of the 
products shared in the OWC in spite of the difficulties in using it, pointed out by the participants. Note that 
mediators could only post products due to its direct contact with students in person meetings. 

The hands-on nature of Popular Education, which stood out in the posts to the OWC, revealed the speech 
of the educators and teams, pointing out the potential and challenges of the territory and the need for change, 
both of the professional’s outlook on his or her own work process and of the ways of teaching and working 
with health in the communities that are, still today, very centered around authoritarian models that defend 
medical authority. 

The qualitative analysis revealed the “Problems Faced” dimension to be the most frequent and, based on 
that, the “Political” category to be the most significant one for the participants, contributing to the 
systematization of the participants’ speech in relation to the feeling of increased precariousness of the work 
of the community healthcare agents and health surveillance agents, which before the PNEPS-SUS was 
limited to the filling out of epidemiological questionnaires, leading to the interpretation that the numeric data 
are more important than people. 

Such result confirms the importance of the PNEPS-SUS as a policy that, in addition to acknowledging 
the importance of such professionals for rendering the concept of health promotion concrete and spreading it 
by means of dialogue (FREIRE, 1997), a step-up from the purely biological concept that works with the 
disease, also focuses on dialogue, care, discussion, shared construction of knowledge, emancipation and 
commitment to the construction of a democratic and popular project (AGADIR and WIMMER, 2013, p. 16). 
In addition to the political dimension, the work also contributed to the systematization of the educational 
dimensions, in the individual and collective understanding of the role of each professional and health 
promoting agent, and thus assigning a new meaning to the practice thereof; of management, insofar as both 
educators and teams join efforts to reach consensual goals, and finally healthcare as it related to the 
singularity of each subject, or how individual experiences his or her illness, considering such act to be the 
result of a process of reflection on the reality that triggers the will to change, to do something, producing new 
meanings. 

Such training contributed to the creation of spaces for the support to online access in the states, in order 
to ensure the access by students and educators. 

For the second stage of the awareness course, the educators will be expected to be able to use the Online 
Community and its functionalities, identifying the OWC as the center of exchange among peers and 
especially of educational support, aiming at contributing with the qualification in Basic Healthcare by means 
of the shared construction of a new outlook on the educational practices in the communities. 

The OWC, as a means for human integration, contributed both for promoting meetings and for recording 
the wealth of human production within a concept of sharing and collective production. The use of Online 
Environments in educational processes already finds, in the literature, solid references to its role as an 
instrument of support to shared construction. This paper, however, sought to evidence the use thereof as an 
instrument that collaborates to the implementation of public policies, having Popular Education as a 
theoretical and practical guideline, within the Brazilian healthcare segment, which by itself evidences its 
innovative nature. 
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ABSTRACT 

The representation of figures in mongean projection (double system planned orthographic projection used in the studies 
of Descriptive Geometry), specially when placed in a particular situation in relation to the projection plans, possesses the 
quality that, through them, the actual dimensions of represented spatial objects can be found directly and without 
additional geometric constructions. However, these representations using orthogonal views present a disadvantage that 
had been observed in which the representation is not always compared to the sight of the actual object. In some complex 
problems, even with great effort of concentration and imagination, the idea of the represented spatial figure cannot be 
reached. 

For current students and future engineers and architects, spatial visualization in perspective and depth is of most 
importance. The binocular or stereo perspective is necessary for the improvement of students’ spatial reasoning. For this, 
stereo visualization programs used in computer graphics can be used. They differ from the method of the cylindrical 
orthogonal axonometric perspective, often taught in the courses of Descriptive Geometry. Thus, it is proposed in this 
work, the use of computational method anaglyphic, in which the binocular axonometric perspective is obtained from the 
mongean projections of the object. The developed program is indicated for the teaching/learning Descriptive Geometry. 


KEYWORDS 

Computer graphics; descriptive geometry; axonometric stereoscopic visualization; geogebra software 


1. INTRODUCTION 

The great evolution of computing and electronics allowed the use of new technological resources for the 
teaching/learning of disciplines on graphics area, such as Descriptive Geometry, Technical Drawing, Space 
Geometry, etc. This way, we can find some studies in the literature that have been developed in the search for 
computational solutions to enhance the teaching of such subjects. [18] proposes an intelligent computing 
environment accessed via Internet, which operates according to the profile of each student. The system 
presents a database of exercises that must be registered by teachers. The environment has not been tested yet. 
Also, we found studies that indicate the potential for stereoscopic computer graphic systems for the 
development of spatial visualization skills [13] and virtual reality techniques to be employed. It is suggested 
to use a stereoscopic display system using polarized projections, in which students would attend classes using 
passive polarized glasses. However, the system was not implemented. Another work of the area has a 
graphical tutorial system, available on the Internet for teaching Descriptive Geometry. The system consists of 
written content, accompanied by illustrative animations. The system, however, is not intended to be used in 
classroom, and does not explore stereoscopic capabilities or real-time interactivity [8], In a more recent study, 
the author has been making use of augmented reality techniques [2], [4] for visualizing and learning of 
conical surfaces [9], [16] implemented the Hipercal3D software, which utilizes virtual reality through 
language VRML, for the viewing and manipulation of objects generated with interactivity using three- 
dimensional visualization perspective and solid modeling for descriptive geometry, but it does not use 
features of stereoscopy. [7] implemented the VirtualGD software, also using the VRML language [19], [20] 
to create a lively and interactive virtual environment, with manipulation of objects in real time, showing the 
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key elements and processes used in Descriptive Geometry. In another subsequent work [6], the authors 
present the use of stereoscopy techniques implemented to VirtualGD. The Descriptive Geometry software 
[17], developed at the University of Lisbon, can be freely downloaded from the website. The program is 
interactive and has offices in both epure and space representation. The system interface is somewhat 
overwhelmed by too much information windows and it is not very suitable for beginning students. Since the 
program AEIOU Descriptive Geometry [1] has many graphics and a clear and friendly interface, it is a 
commercial software. Both programs do not offer resources of stereoscopy. The use of stereoscopic 
techniques has great potential for the improvement of spatial visualization ability, particularly for individuals 
with greater difficulties, because they allow the understanding of spatial (three-dimensional) situations 
presented in exercises, making its development possible. Stereoscopy is related to the ability of depth 
perception. A long time ago, it began to be used in order to facilitate the visualization of complex spatial 
situations and, particularly, in the study of Descriptive Geometry [11]. The first experiments used simple 
techniques like drawings anaglyphies [12]. Nowadays, with the technological development, more 
sophisticated devices are available, which opened new possibilities for the use of these techniques. For 
students, the spatial visualization in perspective and depth is of the most importance and for both the 
perspective using stereoscopy view is necessary. 

In this work, we developed a program, using the GeoGebra educational software [5], in which the 
binocular axonometric perspective of a given object is created from the mongean projections. We observed 
that the most articles published in technical and scientific events in the area are still at the level of academic 
research and are not therefore available for use or do not explore stereoscopic vision. On the other hand, we 
find some commercial products available, but not always perfectly suited to our educational needs. 

We first present the usual matrix method used by computer graphics programs, [3], [15], [21] and that is 
implemented in C language using the OpenGL graphics library, [10], [14] and later we will introduce the 
proposed method, which uses orthogonal axonometric perspective and the GeoGebra software [5], 


2. STEREO ANAGLYPHIC PROJECTION 

Stereoscopic vision consists of depth scene perception by the human visual system. For this purpose, it is 
necessary to capture the image of the object by the two eyes so the brain can estimate the distance between 
them, it means, between the eyes and the object. We will be using the anaglyphic method, in which the image 
of the left eye is colored with cyan and the right eye is colored with red. 

2.1 The Traditional Matrix Method 

The following section presents the method commonly used in computer graphics stereoscopic vision and 
implemented in C language, with the purpose of comparing the figures generated from those obtained by the 
proposed method, presented in section 2.2. 

2.1.1 Visualization in Perspective using the Matrix Method in Homogeneous Coordinates 

Consider the case in which the object is described by points P = (x.y,z) having its coordinates given in 
relation to the orthonormal frame in the world and we want to design it (Figure 1) in a projection plan pp 
located between P and the eye of the beholder e=(e 1 ,e, ,e g ). It is necessary to provide a point a=(a 1: a. .a^) to 
set the direction to where one should look and as the observer's head orientation vector 

u p — (Upl « u pZ '^ps)- 

Thus the intersection of a point P=(x.y.z) of space with the projection plan pp in relation to the observer’s 
eye e is calculated by two coordinate transformations: 
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Figure 1. The projection plan (pp) and the referential of the observer’s eye. 


(I) Frame in the observer’s eye 

Consider the new orthonormal frame (n. v- ST), obtained from the world orthonormal frame by translation 
and rotations, with its center in the observer’s eye and given by 


n = ■ 


-# 
i? = 


u = i? X n 


L«-cJ ' pjrtfipJM 

So by a given point P=(x.y.z) to pass to the coordinates of the eye (X.Y.Z) using homogeneous 
coordinates just consider the transformation 


(X.Y.Z ) = £.-,-) where 

TOW W 



Using the OpenGL library [14] this transformation is given by the command , 

Q lltiiOOfC,4t (®1 J flj * flgj ^pl'^p2' ti )l 

(II) Normalizing the visualization region and leading the observer to infinity 

Consider now the dimensions of the window W (Figure 2) providing horizontal coordinates x=l (left), x=r 
(right), and the vertical coordinates y=b (botton) and y=t ( top) and the region of space where the point lies 
fixing the previous plan by the depth (near) z=-n and depth (far) z=-f. To calculate final coordinates (X.Y), 
of the perspective projection of P on pp, one must now perform the following operations (Figure 2, right): 

(a) shear H to centralize windows W about the depth given by the oz axis. 

(b) adjust the LCD viewing pyramid for a change of scale S for side plans with 45 0 in relation to the 
coordinated plans xz and yz 

(c) finally use a transformation in perspective N taking the eye of the observer to infinity. 

So P'=(NSH)P is obtained in homogeneous coordinates using the matrix 

"* ‘ \ 




MSB = 


2r. 
T— I 
0 




0 

2n 

t-b 

0 


^ ° 
r+b 


r-b 

- tf+Til 

f-n 

-1 


0 

f-n 

0 - 


Using the OpenGL library, [14], this final transformation is presented in the command, 

glFrustum(l, r, b, t, n , /); 
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2.1.2 Stereo Anaglyphic viewing with the Matrix Method 

By designing a point P on a projection plan (pp), according to the observer, we find the PI projection on the 
left eye (le) and the corresponding Pr to right eye (re), which must be on the same horizontal line. The 
distance between the points designed PI and Pr is called Parallax and three situations may occur, depending 
on the relative position of the point P in relation to the projection screen (pp) and the eyes of the observer: 

(i) after the screen where we have positive parallax (Figure 3, left). 

(ii) on the screen where the parallax is null (Figure 3, center). 

(iii) before the screen where we have negative parallax (Figure 3, right). 





v V V V V 


Figure 3. Projections in stereo with positive, null and negative parallax 

In order to perceive the image at depth, it is necessary that each eye sees only its respective image. This is 
done in anaglyphic method as follows using an image editor: 

(a) Name the left eye image of e-image and right eye of r-image. 

(b) Decompose the left eye image in components R (red), G (green) and B (blue) and in order to keep the 
cyan color, the component R should be deleted. 

(c) Decompose the image of the right eye in the R, G and B components and in order to keep the red 
color, it is necessary to delete the G and B components. 

So, by using the anaglyphic glasses with red lens in the left and cyan lens in the right, we will be able to 
visualize the image at depth, as shown in Figure 4 the right image. 



Figure 4. Monocular image (left) and anaglyphic binocular image (right), (LaborMA) 


For the creation of a virtual stereoscopic image, the object must be drawn into a virtual screen 
considering: 

(i) right eye: 

glFrustum(xleftl, xrightl, top, bottom, near, far); 
gluLookAt( re, ra, up; 

and to extract the red component, we must consider the command, 
glColorMask(GL_TRUE, GL_FALSE,GL_FALSE,GL_TRUE); 

(ii) left eye 

glFrustum( xleft2, xright2, top, bottom, near, far); 
gluLookAt(le, la, up; 

and to extract the green (G) and blue (B) components to have the cyan color, we must consider the 
command, glColorMask(GL_FALSE, GL_TRUE, GL_TRUE,GL_TRUE); 

After doing that, one just should superpose the images of each eye and the result will be an anaglyphic 
stereo image. Observe the hexahedron and the FI 17 airplane projections as anaglyphic stereo images in 
Figure 5. 
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Figure 5. Perspectives in anaglyphic stereo using OpenGL of a hexahedron (on the left) and a FI 17 airplane (right) 

2.2 The Proposed Method 

This paper proposes the use of computational anaglyphic method, in which the binocular axonometric 
perspective is obtained from the mongean projections of the object. All these concepts are studied in 
Descriptive Geometry course which emphasizes the educational role of the software. It not only provides the 
appropriate spatial visualization, but also demonstrates and explores the concepts of mongean and 
axonometric perspective projection studied in the course. Its innovative nature comes from the 
implementation of stereoscopic vision from traditional methods used in Descriptive Geometry. The 
developed software uses the free educational program GeoGebra, [5], 

2.2.1 Descriptive Geometry Basic Concepts and the Spatial Visualization Problem 

Descriptive Geometry is an early discipline, specially of engineering and architecture courses. It addresses to 
the representation and solution of problems involving spatial mathematical entities such as points, lines, 
planes, polyhedrons, etc., through its cylindrical orthogonal projections in two planes perpendicular to each 
other, in which one is rotated over the other, with the aid of plane geometric properties, as shown in Figure 6. 
The figure obtained after this revolution is called Epure or mongean representation [10]. 



Figure 6. Revolution of the vertical plane on the horizontal plane (left). Epure or mongean representation (right) 


The distance from one point to the horizontal plane is called quota. The quota coordinate is 
conventionally represented in Epure by the distance from the vertical point projection to the intersection 
planes line. 

The distance from one point to the vertical plane is called afastament. This coordinate is represented in 
Epure by the distance from the horizontal point projection to the intersection planes line, according to Figure 
7. 



-COTA 

AoA2 


- AFASTAMENTO 
AoAl 


Figure 7. Afastament and quota’s first dihedral point. Spatial representation (left) and mongean representation (right) 
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It has been observed that a large number of students have serious difficulties in relation to spatial 
visualization ability, and this is one of the main factors for failure in learning this discipline. The reduced 
workload and the lack of a prior contact with the basic concepts of the discipline are some of the factors that 
contribute to worsen this situation. The non-development of spatial reasoning during the school results in 
students, newcomers to undergraduate courses with difficulties to understand the concepts dealt with in 
disciplines like Descriptive Geometry. In addition to this factor, there is a lack of alternative learning that 
could minimize these difficulties and support the mental abstraction processes necessary for the development 
of spatial visualization skills. The real models are used to facilitate three dimensional understanding. Thus, 
we see the need to seek the use of modern techniques that could minimize this problem. 

2.2.2 Anaglyphic Stereo viewing with the Proposed Method 

The perspective is a graphical representation that shows the objects as they appear to our sight, with its three 
dimensions. The axonometric perspective presented in this work is an orthographic cylindrical projection on 
an oblique plan in relation to the three dimensions of the object being represented. This perspective is 
essentially technical. It is an issue of particular interest on the part of the students of engineering and 
architecture, since they need to develop their capacity for perception and creation of three-dimensional 
objects, in addition to being able of reproducing them in their orthogonal or mongean views, studied in 
Descriptive Geometry. With teaching/learning purposes, more often than not, it is necessary to draw an 
axonometry in order to clarify details that had not been fully understood in the mongean views. The 
axonometric perspective, which construction is based on the inscription of the solid in a trihedron tri- 
rectangle (three right angles), in which only the edges, called axes (axon=shaft, metria=measure), are 
considered. The projection of this trihedron in a plan can lead to the following types of axonometric 
perspective: -Conic: projection from an objective point. 

-Cylindrical projection from a point of infinity 

O Orthogonal: projection lines, orthogonal to the plane of projection. 

O Oblique: projection lines, oblique to the plane of projection. 



Figure 8. Orthogonal axonometric projection elements (left) and the second axonometric triangle (right) 

Figure 8 on the left presents the method of cylindrical orthogonal axonometric perspective, with particular 
distinction to the elements used in its construction, where: 

- O - The vertex of the trihedron goal. For being the closest point of the observer (found at infinity) is always 
visible on projection. 

- x, y and z - axonometric axes 

- x’, y’ and z’ projections of axonometric axes 

- Triangle X 0 Y 0 Z 0 - axonometric or fundamental triangle 

-O’ - orthographic projection of the vertex O. There is a basic Geometry theorem which demonstrates that 
O’ always falls on the orthocenter (date of heights) of the triangle X n Y 0 Z 0 . 

-n 0 - projection plan, always considered in horizontal position. Keeping fixed the trihedron, the lifting or 
lowering of n 0 raises similar axonometric triangles, which results in the same perspective, because this 
depends only on the shape of the axonometric triangle and not on the size of their sides. 

The trihedron formed by orthogonal coordinate systems is projected on the projection plan, in such a way 
that the scale along the axes x, y and z are duly rearranged on the projected axes x ', y ', z ' according to the 
orthographic projection. This procedure is called a ranking of axes designed. Graduating the axes is to score 
on them a specific given unit. In fact, what is sought in order to draw the object perspective is the result of 
this degree in the projections of the axes (perspective unity). 
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As the angle formed by two shafts at the apex is 90°, the triangle is always an acute triangle axonometric 
(presents all three internal angles smaller than 90°). According to the size of the sides of the triangle, the 
cylindrical orthogonal axonometric perspective receives the following names: 

Trimetry or anisometry: If the axonometric triangle is a scalene one, there will be three different 
perspectives of distinct units for the same unit, one for each axe designed. 

Dimetry: If the triangle is isosceles, two perspectives will be alike and one will be different. 

Isometry: it is the most widely used case, if the triangle is equilateral, all three perspectives will be the 
same. 

In order to graduate the projections, we must first determine the real magnitude of the axes, to mark on 
them the given unit and to accomplish that, we must refute a pair of axes on the plane of the painting. After 
that, it is possible to obtain the degree from projections of axes, as presented in Figure 9. 



Figure 9. Graduate axonometric axes 

Using GeoGebra educational software two axonometric projections with two different axonometric 
triangles are built. The triangles have two common vertices and the third one is shifted in space as shown in 
Figure 8 on the right. Using the anaglyphic method, with the figure in red to the right eye and the cyan one to 
the left eye, the projections of the hexahedron and the FI 17 airplane are obtained, as shown in Figure 10. 



Figure 10. Anaglyphic stereo visualization using orthogonal axonometry of the hexahedron and the FI 17 

Figure 11 shows the mongean projection and the stereo anaglyphic axonometric projections of the FI 17, 
as it appears on the screen of GeoGebra. 
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Figure 1 1. FI 17 with mongean projections and anaglyphic axonometric orthogonal projection 

The mongean and perspective projections side by side and interactive, i.e. changing the mongean 
projection the stereo perspective changes automatically. 


3. RESULTS 

The tutorials were developed in the Laboratory of Applied Mathematics (LaborMA) for the Descriptive 
Geometry students of Engineering course, during the year of 2013, as seen in Figure 12. The tutorials were 
designed to test the proposed method with students who faced great difficult in learning this discipline. Two 
groups of fifty students each were provided with the tutorials. It became evident that the stereo perspective 
visualization using the axonometry offered a superior stereo vision at depth in relation to the matrix method 
used as default by the programs of stereo views in computer graphics. Every student who participated in the 
tutorials responded that the proposed method generates figures in stereo with better image quality than the 
traditional method. The use of stereoscopic techniques has great potential for the improvement of spatial 
visualization ability, particularly for individuals with greater difficulties, because they allow the 
understanding of spatial (three-dimensional) situations presented in exercises, making its development 
possible. Another benefit is that the monocular method of axonometry is a tool used in Descriptive Geometry 
for geometric solids visualization and obtained from the mongean projections of the object, also studied in 
this discipline. The software improved both: the visualization ability and the Descriptive Geometry basic 
concepts, which points out its educational role. In addition, we can state that the educational software 
GeoGebra also helps in the learning process, because it is not only easy to handle but also interactive, 
allowing the student to immediate change of point of view, increasing the understanding of the solid, from its 
projections. The discipline Professors observed an improvement in the academic performance of students 
who attended the tutorials and a study to evaluate the educational benefit of the tool is being prepared. 



Figure 12. Students using anaglyphic stereo axonometric orthogonal projections in LaborMA 
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4. CONCLUSION 

This work presented a new methodology for spatial visualization: the use of computational binocular 
axonometric anaglyphic method, in which the axonometric perspective is obtained from mongean projections 
of the object. These concepts are studied in the course of Descriptive Geometry which emphasizes the 
educational role of the software. The method was presented to the students during the year of 2013 and 
provided a greater depth of vision than the method of projective transformations used by default in stereo 
views in computer graphics programs. Its innovative characteristic comes from the implementation of 
stereoscopic vision from traditional methods used in Descriptive Geometry. The software was developed 
using the free educational software GeoGebra. For a future work, we intend to investigate the method using 
conical axonometric perspective. 
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ABSTRACT 

Over the past two decades policies and speculations have been evident about the importance of internet use including 
technologies in education and learning at all levels to individuals and societies. The purposes, theories and ways in which 
learning with technologies ought to be conceptualised and functionalised is generating an increased body of literature. 
With the arrival of Web 2.0 and Semantic Web however not enough is known about the ways in which these online 
technologies interact and may interact with student’s learning experience. There are gaps in our knowledge on the role of 
social networking sites on student learning experiences. Social media tools have become ubiquitous. You can see our 
students use them all the time. Among them most popular tools are Facebook, Wiki, YouTube, bulletin board, Linkedln, 
blogging, and twittering. The advancement of modern technologies tries its best to accommodate the needs from people, 
especially the younger generation. As educators, how can we take advantage of this momentum? With the advent of Web 
2.0 tools, educators are looking to these new technological tools to examine its potential in enhancing teaching and 
learning. While its runaway success as a social networking tool is now renowned, the use of Facebook for educational 
purposes may be considered still at its infancy stage. This paper will bring together recent research findings on how 
learning experience of students at higher and further education level is influenced by the latest development and 
technological advancements of social networking sites. 
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Social Learning Networks, Web 2.0 technologies, e-learning, social learning technologies. 


1. INTRODUCTION 

Social networking sites have seen tremendous growth and are widely used around the world. Nevertheless, 
the use of social networking sites in educational contexts is an under explored area (Ryan et al, 2011). Social 
networking sites (SNSs) have the potential to facilitate interaction, communication, and collaboration, and as 
a result have been prominently featured in discussions centring on the use of technology to support and 
amplify educational endeavours (Greenhow, Robelia, & Hughes, 2009). Empirical research on their role in 
complementary education is limited, even though researchers have identified an accelerating use of social 
software in formal learning contexts (Schroeder, Minocha, & Schneider, 2010). 

Facebook™ is one of the most popular SNSs used today, as evidenced by its 500 million active registered 
users and its rapid growth rate estimated at 105 percent from 2008 to 2009 (Facebook, 2010a; comScore, 
2010). It has been identified as a potential educational tool because it is already used extensively among 
college students. Bowers-Campbell (2008, p. 82) states, “Facebook™ is student-friendly, student centred, 
and student-controlled; the social nature of Facebook™ invites participation instead of mandating it.” In 
addition, Charnigo and Barnett-Ellis (2007) suggest that “by exploring new types of Internet services such as 
Facebook™ instead of quickly dismissing them as ir -relevant, we might learn new ways to reach out and 
communicate with a larger segment of our [academic library] users” (p. 31). Therefore, exploring a SNS tool 
such as Facebook™ in an educational context is both relevant and timely. Our objective is to evaluate factors, 
effects and influences of social networks on students social technology mediated learning. 

The research problem: The use of social media by students has evolved the teaching and learning 
strategies but research on the influences of sharing collaborative tools in classroom such as social networks 
has not been fully investigate. 

The research question: Are social networking software an influential tool in student’s learning? 
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Hypothesis l:“The use of social networks helps as a complementary tool to enhance student’s learning 
experiences”. 


2. RESEARCH SCOPE 

This research study aims to close the gap in the literature by drawing conclusion about the social learning 
technology as a framework that addresses the overlapping pedagogical needs of e-learning and social 
networks with suggestion of methods for implementation of social networks in learning environments. This 
study aims to use mixed methods of qualitative data analysis using case study and quantitative data analysis 
using descriptive statistical methods. 

The literature review conducted identifies the work of many researchers focusing on the use of social 
networks and its effects on learning for many years since the emergence of information and communication 
technology in e-learning environments. Research studies suggest that using social networks with a potential 
to substitute learning management systems has pedagogical, social and technological affordances which 
allows distribution of announcement, sharing ideas and resources and implementation of online discussions. 
However, substitution of a Learning Management System (LMS) with social networks has constraints due to 
lack of support for file format that allows direct uploads and also the lack of organisation in developing 
discussions. Using Facebook for example; as a substitute for LMS enables learners to interact with peers and 
conducts easy communication but it fails to provide a safe environment as student’s perceived privacy is 
decreased. Research performed in this area suggest that for effective use of Facebook in learning, many other 
factors such as effective instructional design, positive instructor’s attitude and strong technical support are 
crucial. (Ozkan & Koseler, 2009). 

One of the research areas that have been neglected is the information sharing feature of social media and 
its implications for learning. Other under searched areas are studies that focus on the challenges faced by 
higher education learners and the cause of learning difficulties with social networking for learning purposes. 
The need remains for qualitative investigations that enables our understanding of how social media impacts 
academic learning. 

Another area which has been under researched is the effectiveness of social networks in higher education 
due to lack of studies that supports the successful implementation strategies of social networks for learning in 
higher education. Studies suggest that many factors need to be giving careful practical attention such as the 
type of learner and also learner’s characteristics need to be considered. Wang, Q., Woo, H. L., Quek, C. L., 
Yang, Y. and Liu, M. (2012). 


3. LITERATURE REVIEW 

Constructivist pedagogy focuses on students constructing knowledge. From a social constructivist (and 
constructionist) perspective, this construction occurs primarily through social interactions (Berger & 
Luckmann, 1966; Vygotsky, 1978; Wertsch, 1986). Web 2.0 collaborative technologies promote social 
interaction. They allow students’ work to be read and commented on by a larger participant audience than 
afforded in traditional constructivist education. Using collaborative technologies, students can communicate 
with classmates as well as with others around the world. Comments made by this diverse, participatory 
audience often generate discussions that enhance learning. 

Web 2.0 tools can promote user participation and knowledge production and thus fit well with social 
constructivist pedagogical theories. These tools have the potential to transform classes from teacher-centric, 
transmission instruction to social constructivist, student-participatory approaches, from individual-focused 
pedagogies to learning community approaches. Even as constructivism has and continues to be a main focus 
of learning theorists, the technological tools used in education have become increasingly powerful and 
crossed the gulf between day-to-day life and education. These tools relate to one another under the term 
technology and are the subject of this study i.e. Social Learning Technology. 

Social Learning Technology has been embraced by some and disgraced by many, yet today’s digital 
natives navigate virtual worlds without hesitancy or misgivings. Research suggests that “Students are far 
more technologically savvy than the institutions that support them” (Desai, Hart, & Richards, 2008, p. 329). 
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This poses a problem as teachers try to reconcile personal constructivist pedagogies with a tool they are 
unaccustomed to or intimidated by. Yet, it’s this very social learning tool which opens the door to new and 
innovative applications of constructivist teaching and learning methods. According to Desai, Hart, and 
Richards (2008), “The vast amount of information that computers supply on a daily basis has allowed 
teachers and students new ways to explore education compared to ordinary instructional tools” (p. 329). 
Social Network Technology offers flexibility and adaptability reflective of pedagogies across various 
learning models based in constructivism. 

According to Wenger (2002), “The community creates the social fabric of learning” (Wenger et al., 2002, 
p. 28). The community is a group of people who learn and interact together, building relationships that result 
in a feeling of belonging and mutual commitment (Wenger, 1998). 

Brown (2001), states that interaction is at the heart of communicative competence. When a learner 
interacts with another learner he/she receives input and produces output. Young people can learn more 
readily when communicated information is tangible and directly accessible to their senses — visual, auditory, 
tactile, and kinaesthetic. With experience, learners grow in their ability to understand abstract concepts, 
manipulate symbols, reason logically, and generalize. 

Communication provides opportunities for learning. In the cognitive learning tradition, participating with 
others in groups can provide an opportunity to generate explanations, which results in deeper individual 
cognitive processing and hence, better learning (Chi, de Leeuw, Chiu, & LaVancher, 1997). 

Research and practice indicates that communities provide fertile ground for socio -cultural appropriation 
(adopting expert practices through social processes) as well. (Lave & Wenger, 1991) discuss the reciprocal 
relationship between communities and learning. 

Virtual learning environments (VLEs) are widespread in higher education today, typically used 
to deliver instructional materials and facilitate communication within a course. Remote delivery of 
courses became a viable option with the World Wide Web and online education of various sorts is now 
routinely available to vast numbers of students (Peffers & Bloom 1999; Alexander 2001; Chen & Dwyer 
2003). Various terms have been used to label or describe forms of education supported by information 
technology. These include e-learning, web-based learning, online learning, distributed learning and 
technology-mediated learning; with e-learning probably the term most commonly used to describe education 
and training that networks such as the Internet support. A virtual learning environment (VLE) is an 
information system that facilitates e-learning. VLEs process, store and disseminate educational material and 
support communication associated with teaching and learning. 

In terms of using new social networking tools and environments as a supplement to current 
Classroom teaching, one of the initial challenges will be to develop learning activities that use these tools 
to leverage student motivation and learning. Researchers argue that this is made more challenging as children 
of the “digital age” have different needs than previous learners (Brown, 2002; Schrum & Solomon, 2007; 
Green & Hannon, 2007). Pedagogical methods may need to be adjusted in order to allow for students to 
increasingly learn from each other. Green & Hannon (2007, pg. 26) indicate, “with the advent of blogging 
and tools such as Wikipedia, young people are just as likely to seek feedback from their peers and strangers 
as they are from teachers and parents. This has led to the blurring of the boundaries between expert and 
amateur, friend and mentor.” 


4. METHOD 

In this study the interpretive research method with the aid of pilot study method was used to investigate the 
use of social networks in classroom. The constant comparative method (Glaser & Strauss, 1967) was used to 
analyse online survey responses, arriving at categories and data patterns. We engaged in open coding of all 
data in order to identify emerging patterns with regards to student’s online learning experiences on the same 
course of study. The patterns were compiled and codes confirmed across all participants. Open coding of data 
resulted in patterns that could be grouped into themes. Learners found their interactions with others were 
important in helping them make Sense of the subject matter and reported that these interactions extended 
their learning. 

The ease with which participants were able to communicate was also deemed to be important to the social 
connectivity. 17 


224 



International Conference e-Learning 2014 


The participants in this study included students at a Higher Education Institution and its associated 
Further Education College studying on degree and advanced level courses. Learner’s achievement data and 
online performance in a pilot study were collected and compared in order to establish the match between 
student’s level of attainment and their online performance while using social networks such as Facebook as a 
complementary study platform. 

Second series of pilot study were conducted in order to evaluate the role of social networks and its effects 
on learning outcomes through enhancement of communication methods. Both qualitative and quantitative 
data were gathered and analysed with some suggested guidelines. Experiments were designed according to 
participant’s level of study. The first pilot study involved three groups of students at Middlesex University on 
the first year of an I.T. Degree course on a study period of two weeks. The second pilot study involved four 
groups of students at an associated FE college on an advanced level ICT course on a study period of two 
months. 


5. DATA COLLECTION 

The survey was primarily used to collect responses to open-ended questions that sought feedback about (a) 
student experiences, and (b) specific learning activities and attributes of the course. The first study survey 
consisted of questions related to student’s learning activities, tutor’s teaching style and learner’s attitude 
towards their tutor, learner’s assumptions and regards towards using social networks as a tool and its future 
potential. 

Data collected related to participant’s response to questions. Data was coded and themes identified. Also 
data related to participant’s predicted achievement grades were analysed against the data collected related to 
some open ended questions that relates to learner’s feedback on their instructor’s level of support and 
tutoring style. This statistical analysis identified that a positive correlation coefficient identified through this 
study suggest that there is an strong relationship between participant’s level of achievement and their 
preference for their tutor’s teaching style and their attitude towards using Facebook as a collaborative tool 
that enhances student’s learning experiences. 


Table 1 . Pilot study process 
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5.1 Findings 

The noticeable theme is that almost all members of these groups liked their tutors teaching style and their 
responses indicate that they were satisfied with the level of feedback and the support received from their 
tutor, although some results indicate that participants did not take part in the process; either because they did 
not regard the question as relevant or supportive. Another noticeable trend is that almost all participants 
found the creation of a Data Flow Diagram (DFD) a difficult task, as well as the creation of an Entity Life 
History (ELH), identifying entities for the Business Information Systems (BIS) module and drawing 
relationships. Another trend was the lack of or the irresolute responses as to whether the groups regarded 
Face Book (FB) to be a medium for enhancement of their learning with BIS module. 

In the direct question and answer of 42 participants, 81% stated that they like using FB and 52% said it 
does facilitate communication and connection between people. Around 14% thought it causes distraction 
from studying and 10% said they did not like doing difficult things using FB while also 10% did not know 
FB’s potential use. Around 7% thought they do not like the lack of privacy and also around 5% said that FB 
pages lack required structure for learning. Less than 1% said they do not like using FB at all. 

5.2 Interviews 

Interviewing participants in the three groups identified some themes. It can be seen from the pattern of its use 
that those participants who favoured and enjoyed using Face Book as a social media also used it for 
establishing communication and connecting to others within the same group or other groups outside 
university. A large proportion of those who favoured using Face Book also regarded its use as a positive 
aspect that enabled them to use instant messaging, chatting, tagging photos, watching videos etc. A 
comparison of the themes indicates that a user with a high positive response also corresponds to a low 
negative aspect. Amongst the implicit themes, there are a few anomalies that do not follow the pattern. For 
example user25 has a high positive response and also a relative high negative aspect in comparison to the 
other users. This could be an indication of the user’s balanced view of Face Book use. 

This implies that this participant has either informed views on positive and negative aspects of Face Book 
or that is in favour of use of this media on moderation. A small proportion of approximately 4.8% of the 
users were able to lend themselves to both the positive and negative aspects of Face Book use; obviously 
indicating that the vast majority were very one sided. 

5.3 Pilot Study Process Analysis 

Participants in this experiment belong to two groups of first year and second year students. The unit delivers 
content on learning about computer systems that include both hardware and software. This group learns how 
the components of motherboard work and relate to each other in operating the hardware and also how the 
different software of a computer system including operating system and application software complement the 
tasks of the hardware. Students are assessed on the knowledge of these topics by completing tasks within 
projects that include individual and groups work and submit their reports through the Virtual Learning 
Environment (VLE) environment on the college’s website. Another module undertaken by the second year 
groups is software design in which students learn the history of development of different programming 
languages along with learning how to program within a fourth generation language such as visual studio. 
Assessment is based on demonstrating their knowledge of history of different programming language 
generation and classification, characteristics and features as well as development; testing and documentation 
of software designed and programmed using a 4GL. Majority of student’s work is based on independent 
learning and assessment while a small fraction of each module is based on group work and presentation. 
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Table 2. Mapping of Pilot study processes 

Units 

How FB can help with 
its learning 




5.4 Focus Groups 

This study examined the effectiveness of using social network Facebook for improved communication by 
learners on advanced level of ICT course. All students who had completed their first semester at college were 
given the opportunity to participate in this study. Almost all students who were invited took part in this 
activity with a minority of those who had a more isolated presence and were not welcoming interactions with 
other classmates or tutor. 

Once member of groups were selected and recruited, participants were informed that one aspect of the 
study would require them to photograph themselves and upload it on their profile. These photos were then to 
be uploaded to the Facebook research group created specifically for this study, and they were instructed to 
make brief comments about their posts. What transpired from there became a wealth of information from 
which it was possible to collect valuable data to build upon and report back as findings. The group was 
private, so activity such as photo uploads and messages was only visible to the members and administrators 
of the group, allowing participants to speak freely without sharing these updates with their entire Facebook 
network. This feature is also beneficial in that the group members can see and comment on each other’s 
uploads, which creates discussion and an interesting dynamic to observe where tutor asked to perform similar 
tasks react to each other’s interpretations of the task. What’s also great about it is that we can communicate to 
the entire group at once to notify them of study changes, additional aspects of the study, let them know their 
posts are well received, and so on. 

The biggest benefit is that Facebook’ s ubiquitous popularity makes it a simple tool for the participating 
groups to use - browsing, receiving messages, uploading photos and many other characteristics of the site are 
fairly common to most people, especially among the demographic we were studying (ages 16-24). The rise of 
smart phones and the many options for mobile uploads to Facebook (via text, email or through a mobile app) 
allowed participants to upload their content live from the location wherever convenient. As a result, this 
process was not cumbersome to participants and made it that much more enjoyable for them. 

5.5 Observations 

There was a diversity of expected and unexpected outcomes. Most students showed somehow an uncertain 
view on use of Facebook in presence of their tutor and although some seemed more forward for the idea, few 
could not understand the relevance at first. At first very few students communicated that they do not use this 
media at all and therefore lacked the required ability to enable them make a comment on Facebook’ s 
suitability for its purpose. It was interesting to see the same few participants had a totally different view 
following couple of sessions using Facebook for the purpose of this experiment. 
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The online interaction on Facebook indicated that instant messaging feature of communication to be a 
more favoured aspect of its use with these participants. Many students favoured the ubiquitous characteristic 
of Facebook and accessing this social network on Mobile technology. Many students were concerned about 
privacy issue with Facebook use and therefore did not regard its use for learning and communication to be a 
favoured feature to the point that even the issue of separating personal status from educational status on 
Facebook did not seem an attractive alternative to them. The major problem reported by some students was 
the issue of distraction and diversion from the focus of activities due to the social nature of Facebook. 


6. DISCUSSION 

The asymmetric information flow is a dominant characteristics of personalised platforms created. Web 2.0 
creates the use of centralised nodes for social networking users. Through the shared online social networking 
sites, web users interact and exchange information with others which mirrors real life networks between 
people. Use of Web 2.0 emphasises collaboration, knowledge sharing and conversion amongst people. This is 
seen as a superior characteristic than just simply having a discussion within a public domain. The focus of 
Web 2.0 is on user driven content in which user messages do not represent goal orientated communication. 
As the concept of “Web 2.0” has quickly gained widespread popularity, a new generation of E-learning 
associating with Online Social Networking sites will impact students’ learning experience. Social networking 
software has provided many features that can serve the learning sector in many ways. 

(Howe, 2006) claims that users recognise the web as first a service delivery platform and second as the 
collective wisdom of the crowd. These are the two important attributes of Web 2.0 technology. Facebook as a 
strong example of Groundswell phenomenon (Li & Bernoff, 2008), exists both as a service delivery platform 
and also as a stockpile for collective groupthink. Facebook promotes the user as a centralised node around 
which information revolves. The use of Web 2.0 and social networking technology represents an attractive 
force that empowers people’s ability to communicate and foster social connections with one another to create 
a variety of communities of practice. 

For the purpose of next phase of this study, a series of pilot studies are considered. Under the pragmatic 
research approach and using mixed methods of qualitative and quantitative data collection, learner’s online 
response to surveys, interviews, and questionnaires will be logged as well as observation of their methods of 
communication, comment, tags, message and online pokes and chats will be collected. Also quantitative data 
related to performance and grades of these candidates using online media against these data obtained through 
traditional teaching methods are to be analysed through simple statistical methods, and interpreted. 


7. CONCLUSION 

The social learning technology framework proposed in this research study is a solution based on Web 2.0 
tools and technology which defines a model for its role and application. It is aimed that this framework also 
will address the needs of its implementation in higher education and reach beyond its needs to persevere its 
integrity of use. The SLT framework focuses on social aspects evaluating group dynamics based on 
electronic learning and communication styles. This framework relies on technology support with the advent 
of web 2.0 technology tools and advanced e-learning systems. 
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ABSTRACT 

Web 2.0 technology has drawn much attention recently as a fascinating tool for Information Literacy Instruction (ILI), especially in 
academic libraries. This research was aimed to investigate the implementation of Web 2.0 technology for ILI in Thai university 
libraries, in terms of information literacy skills being taught, types of Web 2.0 technology that were implemented, ways of 
implementing Web 2.0 technology, and problems in implementing Web 2.0 technology. Additionally, in case of the university 
libraries which did not apply Web 2.0 technology to their ILI, the research also explored their reasons, implementation plan, and 
factors important to their decisions on the implementation of Web 2.0 technology. Despite the research limitation in the context of 
Thai higher education, the research should be helpful to fulfill the research gap, and, particularly, to provide other university libraries 
with interesting suggestions on the implementation of Web 2.0 technology for ILI, including the reinforcement of the implementation. 
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1. INTRODUCTION 

Information Literacy (IL) has been the term coined since 1974 by Paul G. Zurkowski (Grassian & Kaplowitz 2009, p. 3). 
Now, it is still recognized and so important that it is a prerequisite for participating effectively in the Information 
Society, and is part of the basic human right of life long learning” (Thompson 2003). Many have defined this term but 
the one most commonly accepted is defined by the American Library Association (ALA) (Campbell 2004). It indicates 
that an information literate can find, evaluate, and use information effectively no matter where the information comes 
from (American Library Association 1989). 

In academic setting, these three IL standards are best known (Webber 2008, p. 41): the standard issued in 2000 
(ACRL 2000) by the Association of College and Research Libraries (ACRL), the standard issued in 1999 (SCONUL 
1999), which was recently revised in 2011 (SCONUL Working Group on Information Literacy 2011), by the Society 
of College, National and University Libraries (SCONUL), and the standard issued in 2004 (Bundy 2004) by the 
Australian and New Zealand Institute for Information Literacy (ANZUL). A correlation between these standards does 
exist as ANZIIL’s standard is developed from ACRL’s standard (Bundy 2004, p. 3) whereas SCONUL’s recent 
standard is comparable to ACRL’s standard (Martin 2013). 

In this environment of information overload, IL is essential to an individual both as a person and a citizen. As a 
person, an information literate is able to be a lifelong learner who can determine information needs as well as locate, 
access, evaluate, and use information effectively and efficiently for personal, academic, and working purposes (Detlor 
et al. 2011, P- 572). As a citizen, IL contributes to an active participant who is able to gain and express information 
powerfully (Livingstone, Van Couvering & Thumim 2008, p. 105). 

Since IL is a critical issue, librarians consider IL development as one of their key roles (Ferguson 2010) and 
Information Literacy Instruction (ILI) as one of their main services, particularly in academic libraries (Hart 2010, p. 
181). Because today’s society is networked, new Information and Communication Technologies (ICTs) should be 
incorporated into ILI (Fernandez- Villavicencio 2010, p. 128). Increasingly, a technology that has drawn librarians’ 
attention recently is Web 2.0 technology (Luo 2010, p. 32). 
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Web 2.0 technology refers to web-based technology that emphasizes users, interoperability, communication, 
collaboration, and information sharing (Carpan 2010, p. 106; Whittaker & Dunham 2009, p. 51). The types of Web 2.0 
technology that are among the most popular applications (Kelly 2008, p. 22) which can be useful to ILI are Blog, 
Instant Messaging (IM), Media Sharing, Podcasting, Social Bookmark, Social Networking, Virtual World, and Wiki 
(Fernandez- Villavicencio 2010, p. 131; Godwin 2008a, pp. 168-174; Grassian & Kaplowitz 2009, pp. 304-307). 

Despite numerous positive potentials of Web 2.0 technology to ILL, as illustrated in some literature (Bobish 2011, 
pp. 56-63; Click & Petit 2010, pp. 138-141; Godwin 2007, pp. 105-1 10), it is a waste of opportunity that the technology 
offers (Hart 2010, p. 181) if no or only a few university libraries implement Web 2.0 technology for ILI. Additionally, 
since specific cases and the focus on one type of Web 2.0 technology prevail the literature on Web 2.0 technology and 
its adoption (Luo 2010, p. 33; Magnuson 2013, p. 250), it is necessary that a study revealing a holistic view be 
undertaken to fill the research gap. This is especially important in the context of Thai university libraries where most 
studies found are about the assessment of students’ information literacy and there is currently no study addressing the 
application of Web 2.0 technology to ILI. 

This research was aimed to investigate the implementation of Web 2.0 technology for ILI in Thai university 
libraries, in terms of information literacy skills being taught, types of Web 2.0 technology that were implemented, ways 
of implementing Web 2.0 technology, and problems in implementing Web 2.0 technology. Additionally, in case of the 
university libraries which did not apply Web 2.0 technology to their ILI, the research also explored their reasons, 
implementation plan, and factors important to their decisions on the implementation of Web 2.0 technology. It was 
hoped that the research should contribute to the implementation of Web 2.0 technology for ILI, including the 
reinforcement of the implementation, especially in university libraries. 


2. WEB 2.0 TECHNOLOGY AND ILI 

Since the introduction of Web 2.0 technology in the academic sphere, a variety of literature has valued its advantages to 
learning (Farmer, Yue & Brooks 2008; Luce-Kapler 2007; Minocha 2009; Vaughan 2010). In ILI, the technology has 
also enticed academic libraries for many reasons. 

Firstly, today’s students are digital natives, Internet generation, millennials, and NetGen (Kent 2008), characterized, 
for instance, by their natural use of information and technology as well as preference of collaboration, teamworking, 
and social networking (Godwin 2008b, pp. 5-6). They have widely used Web 2.0 technology (Collis & Moonen 2008, 
p. 95). Compared to their elders, they were the group doing these activities more: using social networking sites and IM, 
reading blogs, working on own blog, downloading podcasts, and participating in virtual worlds (Zickuhr 2010). In 
Thailand, the age group of 15-24 years also used the Internet most (National Statistical Office of Thailand 2011). 
Specifically, in a tertiary institution, it was found that the majority of undergraduates knew and ever used many types of 
Web 2.0 technology, namely, IM, Media Sharing, Podcasting, Social Networking, and Wiki. Also, the majority of them 
needed to use Web 2.0 technology for learning (Sawetrattanasatian 2013). As a result, implementing Web 2.0 
technology for ILI to teach this user group should be useful as it was a way to reach them where they were acquainted 
to and would promptly engage in (Williams 2010, p. 161). Importantly, despite their familiarity with technology, it was 
described in the library and education domain that students were generally lack of information literacy skills (Detlor et 
al. 2011, p.573). 

Secondly, Web 2.0 technology could be diversely applied to ILI as depicted in some literature (Bobish 2011, pp. 
56-63; Click & Petit 2010, pp. 138-141; Godwin 2007, pp. 105-110; Godwin 2008a, pp. 168-174; Grassian & 
Kaplowitz 2009, pp. 304-307; Luo 2010, pp. 34-36). On the issue related to ways of implementing Web 2.0 for ILL, the 
technology could be used for instructor’s (instructors’) own personal purposes (e.g. information sources for teaching, 
organization and management of teaching materials); examples to illustrate the concept of information literacy to 
learners; distribution of teaching content, news relevant to the instruction, and resource/information access to learners; 
cases, which learners will study and complete their assignments, so that they can understand the concept of information 
literacy; means of interaction, between instructor(s) and learners, and between learners; and places where learners share, 
work, and present their works (Deitering & Gronemyer 2011; Fernandez- Villavicencio 2010, p. 133; Godwin 2007, p. 
1 10; Luo 2010, pp. 34-36; Whittaker & Dunham 2009, p. 57). 

Finally, adopting Web 2.0 technology into ILI could support the pedagogy of constructivism which was “... 
increasingly popular among educators ...” (Farkas 2012, p. 86). This is because while using Web 2.0 technology, 
learners are allowed to collaboratively create, gain, and share information. They could act as an active agent, not only a 
passive recipient (Dunaway 2011, p. 155). In brief, it could be summarized that Web 2.0 technology could reinforce 
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active learning which could also be regarded as learner-centered learning in which learners could get involved 
(Grassian & Kaplowitz 2009, p. 223) and this is “...an important ideal, both in educational institutions generally as 
well as in library settings . . (Jacobson 2009, p. xxvi). 

In the circumstance of academic, medical, and research libraries, only 18% of the articles, published during 2006- 
2011, were dealing with Web 2.0 tools used in user education and information literacy (Gardois et al. 2012, p. 90). 
Additionally, most previous studies on the integration of Web 2.0 technology into ILI were about individual types of 
the technology (Luo 2010, p. 33; Magnuson 2013, p. 250). There was not much empirical research, notably in 
academic libraries where the background of the current research was based. However, since grasping a comprehensive 
view of a phenomenon was indispensable to the insightful understanding, some stimulating studies did still exist. 

Luo (2010) examined the current practice of Web 2.0 integration in information literacy instruction and found that 
librarians surveyed applied Web 2.0 technology in teaching IL. Their adoption of Web 2.0 tools could be divided into 
three levels; Level 1 - Using Web 2.0 tools to organize and manage course-related material for librarians’ own 
purposes. Level 2 - Using Web 2.0 tools to facilitate the delivery of content to students, and Level 3 - Using Web 2.0 
to illustrate IL concepts. Among these levels. Level 2 was found as the framework that most libraries applied for Web 
2.0 integration into ILI. The study also revealed that the challenges librarians had to overcome were technical 
challenges, online vandalism, and students’ preconceptions of Web 2.0. 

Daniels & Huxor (201 1) investigated the current level of Web 2.0 usage among academic liaison / subject librarians 
in IL programmes. RSS feeds. Weblogs, and Wiki were the most commonly used applications whereas Social 
Bookmark was used limitedly. 

Magnuson (2013) examined how Web 2.0 tools in an online information literacy instruction course could correlate 
to ACRL’s IL standard. Positively, it was reported that Web 2.0 could enhance all five IL standards. She also identified 
five themes in which Web 2.0 enhanced learning: sharing and collaboration, organization, creativity and enjoyment, 
catalyst for discussion, and learning about educational technology. 

Regarding Thai literature, at the time when the current research was conducted, there was no study interrogating the 
implementation of Web 2.0 technology into ILI. There were only some studies reflecting the issues of IL development 
in academic libraries (Khamhomkun 2011; Sirichai, Techamanee & Treewanich 2010; Tuamsuk 2013). 


3. METHODOLOGY 

In this survey study, a questionnaire responding to the research objectives was developed as the research instrument. It 
was pretested by a random of ten library heads or library staff members who were responsible for information literacy 
instruction in ten faculty libraries, located in the universities from different geographical regions of Thailand. 

After the pretest and revision, during August and October 2013, the final version of questionnaire was distributed 
by mail to collect the data from the library director/library head/library staff member who was responsible for 
information literacy instruction in all 127 Thai university central libraries. The questionnaire was divided into three 
parts: Preliminary data on ILI of the library (Part 1), Implementation of Web 2.0 technology for ILI (Part 2), and Non- 
implementation of Web 2.0 technology for ILI (Part 3). In regard to Part 2, ACRL’s standard (ACRL 2000) was used 
for the question on information literacy skills being taught since it is in accordance with the SCONUL’s and ANZIIL’s 
standards, as mentioned above. Also, it is the influential standard in higher education (Magnuson 2013, p. 245). Of the 
questionnaires sent, a total of 93 were returned and usable for further analysis, so the overall response rate was 73.23%. 
Then, the data collected were analysed using frequency, percentage, mean score, and standard deviation. 


4. FINDINGS AND DISCUSSION 

After analyzing the collected data, the preliminary data on ILI of these respondents revealed that most university 
libraries provided information literacy instruction (76.34%). Among them, most had librarians as those who were 
responsible for ILI (90.14%) and had undergraduate students as their target audience (97.18%). The approach that most 
libraries took to providing ILI was independent instruction offered by the library as supplemental instruction to a 
curriculum (84.51%). Generally, the majority of university libraries (60.56%) did not implement Web 2.0 technology 
whereas the rest (39.44%) did implement it. To explain this, apart from what they answered for their reasons on the 
non-implementation, it may be due to the claim that “. . . librarians in developing countries are still at the early stage of 
getting themselves familiarized with the 2.0 tools . . .” (Esse 2013, p. 183). 
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4.1 Implementation of Web 2.0 Technology for ILI 

This section would describe the major findings found for Thai university libraries who implemented Web 2.0 
technology for ILI (39.44%) as follows: 

4.1.1 I nf ormation Literacy S kil ls Being Taught 

The majority of university libraries implemented Web 2.0 technology for ILI to teach all five skills as displayed in 
Table 1. The findings confirm Bobish’s (2011) and Magnuson’s (2013) literature describing the possibilities of Web 2.0 
to foster all skills of ACRL’s standard. Regarding the standard which most respondents (96.43%) applied Web 2.0 
technology, it was Standard 2: Accesses needed information effectively and efficiently. This is in line with the research 
findings indicated that most Thai university libraries promoted accessing needed information effectively and efficiently 
(Khamhomkun 2011). 

Table 1. Information Literacy Skills Taught by Implementing Web 2.0 Technology 


Information Literacy Skills Percentage 

Standard 1 : Determines the nature and extent of the information needed 85.7 1 

Standard 2: Accesses needed information effectively and efficiently 96.43 

Standard 3: Evaluates information and its sources critically and incorporates selected information into his or her 64.29 

knowledge base and value system 

Standard 4: Individually or as a member of a group, uses information effectively to accomplish a specific purpose 7 1 .43 

Standard 5: Understands many of the economic, legal, and social issues surrounding the use of information and 71.43 

accesses and uses information ethically and legally 


However, it is also noteworthy that Standard 3: Evaluates information and its sources critically and incorporates 
selected information into his or her knowledge base and value system was the standard that least university libraries 
(64.29%) implemented Web 2.0 technology although information evaluation was identified as a concept “... better 
illustrated with reference to the Web 2.0 technology” (Luo 2010, p. 36). More importantly, in today’s world of 
abundant information, it is the skill that must be increasingly highlighted (Farkas 2012, p. 90). 

4.1.2 Types of Web 2.0 Technology That Are Implemented 

The majority of university libraries implemented four types of Web 2.0 technology for ILI as presented in Table 2. 
They were Social Networking (92.86%), Media Sharing (78.57%), Blog (64.29%), and Instant Messaging (IM) 
(57.14%). This may be because these types of Web 2.0 technology were quite prominent in the library realm. Social 
Networking, Blog, and IM were among the popular technologies for USA and UK librarians (Arif & Mahmood 2012, 
p. 475). Similarly, Social Networking and Blog were among the most used social media in European librarians 
(EBSCO 2010). In regard to Thailand, it was also found in a study on Web 2.0 technology uses and needs of state 
university libraries that Social Networking, Blog, and IM were the top three types of Web 2.0 technology that most 
state university libraries used (Paphatsurichote 201 1). 


Table 2. Types of Web 2.0 Technology Implemented for ILI 


Web 2.0 Technology 

Percentage 

Web 2.0 Technology 

Percentage 

Blog 

64.29 

Social Bookmark 

21.43 

Instant Messaging (IM) 

57.14 

Social Networking 

92.86 

Media Sharing 

78.57 

Virtual World 

10.71 

Podcasting 

35.71 

Wiki 

25.00 


These four types of Web 2.0 technology could be implemented into ILI variedly as demonstrated in these examples. 
Social Networking, such as Facebook, Linkedln, and Myspace, could be applied to ILI for illustrating the need of being 
able to consider the quality of information, promoting IL training activities, and communicating with patrons 
(Fernandez- Villavicencio 2010, p. 133; Luo 2010, p. 36). Media Sharing, such as Flickr, Picasa, Slideshare, and 
Youtube, could be applied to ILI for serving as a source for teaching, distributing instructional media, and sharing 
students’ content (Luo 2010, p. 35; Whittaker & Dunham 2009, p. 57). Blog, such as Blogger, GotoKnow, and 
Wordpress, could be applied to ILI for presenting course-related content, supporting class interaction, and providing 
subject-related blogs that students could explore to have better IL understanding (Deitering & Gronemyer 2011; Luo 
2010, p. 34-35). Finally, apart from functioning as a means to inform learners news relevant to the instruction or a place 
where learners could work and share, IM, such as, Google Talk (Google+ Hangouts in 2013), MSN Messenger, Skype, 
and Yahoo! Messenger), could be principally applied to ILI for communication (Godwin 2007, p. 1 10). 
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4.1.3 Ways of Implementing Web 2.0 Technology 

The majority of university libraries used almost every type of Web 2.0 technology for distribution of teaching content, 
news relevant to the instruction, and resource/information access to learners, namely. Blog (83.33%), Instant Messaging 
(62.50%), Media Sharing (77.27%), Podcasting (60.00%), Social Networking (84.62%), Virtual World (66.67%), and 
Wiki (57.14%). The exception was Social Bookmark (50.00%) but the percentage was still the highest compared to 
other ways university libraries implemented Social Bookmark into ILI. 

The findings correlated to what Luo (2010) found in her study that most librarians adopted Web 2.0 tools in ILI to 
facilitate the content delivery to students. Nevertheless, this way of implementing Web 2.0 technology could not yield the 
most benefits the technology could offer (Dunaway 201 1, p. 152) as it is deemed as passive instruction where learners 
are the consumers, not active participants or creators. Consequently, they do not practice the activities of high order 
thinking involved in active instruction, which could result in preferable learning outcomes (Detlor et al. 2012, p. 156). 

4.1.4 Problems in Implementing Web 2.0 Technology 

According to the 5-Likert scale (most problematic, very problematic, quite problematic, not very problematic, least 
problematic), university libraries considered all four types of problems as quite problematic respectively: problems 
relevant to operation (3.22)*, problems relevant to learners (3.10), problems relevant to instmctor(s) (2.93), and other 
problems (2.90). 

Concerning the sub-problems, all were rated as quite problematic except the one receiving the highest mean score 
under problems relevant to operation which was viewed as very problematic. The sub-problems receiving the highest 
mean score under each type of problem were as follows: The library did not have enough staff to implement Web 2.0 
technology into ILI (3.65) for problems relevant to operation, learners did not have enough knowledge and skills to use 
Web 2.0 technology (3.15) for problems relevant to learners, instructors) did not have sufficient time to implement 
Web 2.0 technology into ILI (3.1 1) for problems relevant to instructor s), and there were some risks in using Web 2.0 
technology, such as the issues of authority, security, and privacy (3.28) for other problems. 

Compared to all other sub-problems, that the library did not have enough staff to implement Web 2.0 technology 
into ILI is vital since it received the highest mean score and was the only sub-problem rated as very problematic. In 
some studies on IL reinforcement in developing countries (Baro & Keboh 2012; Sirichai, Techamanee & Treewanich 
2010), inadequate number of competent or trained staff was also mentioned. Additionally, this was in accordance with 
the findings of the university libraries who did not implement Web 2.0 technology for ILI, described in the next section. 
This is because most of them did not implement Web 2.0 technology for ILI because they had a limited number of staff 
and existing staff already had a lot of works to do. Also, the sub-problem might be directly relevant to two internal sub- 
factors, important to the decisions on implementing Web 2.0 technology for ILI. They were instructor’s (instructors’) 
knowledge and skills to implement Web 2.0 technology into ILI and adequacy of library staff, which were ranked 
among the internal sub-factors receiving the first three highest mean scores by those who did not implement the 
technology. 

Respecting the problems relevant to learners, that learners did not have enough knowledge and skills to use Web 2.0 
technology is interesting as, generally, it is assumed that this generation of learners should be digital natives. However, 
they might have different technology literacy and competency (Farkas 2012, p. 88). The sub-problem was also aligned 
with Luo’s findings that a challenge librarians should overcome when adopting Web 2.0 into ILI was students’ 
unfamiliarity with the Web 2.0 technology (Luo 2010). Learner’s knowledge and skills to use Web 2.0 technology was 
also a highly-rated external sub-factor, important to the decisions on implementing Web 2.0 technology for ILI, 
described in the next section. 

Time is needed for librarians who teach IL to . . develop appropriate content, instruct, solicit feedback, respond 
and engage students in an active learning process . . .” (Owusu-Ansah 2004, p. 9). This statement is still true to the 
implementation of Web 2.0 technology for ILI and it might explain why the sub-problem that instructor(s) did not have 
sufficient time to implement Web 2.0 technology into ILI received the highest mean score for problems relevant to 
instructors). This might also be the result of librarians’ work overload and staff insufficiency. 

Finally, another sub-problem receiving the highest mean score that could hinder the implementation of Web 2.0 
technology for ILI was the technology itself as there were some risks in using it, such as the issues of authority, security, 
and privacy. This might be due to its open unmonitored environment (Godwin 2007, p. 106) and its general nature of 
external hosting (Kelly 2008, p. 23). 


The number in parentheses is the value of mean. 


234 



International Conference e-Leaming 2014 


4.2 Non-Implementation of Web 2.0 Technology for ILI 

This section would describe the major findings found for the majority of Thai university libraries who did not 
implement Web 2.0 technology for ILI (60.56%) as follows: 

4.2.1 Reasons of Non-Implementation 

Most university libraries did not implement Web 2.0 technology for ILI because they had a limited number of staff and 
existing staff already had a lot of works to do (72.09%). In fact, even without the adoption of Web 2.0 technology, lack 
of staff was also another existing challenge librarians faced in ILI as stated in some literature both in developed and 
developing countries (Detlor et al. 2012; Idiodi 2005; Lwehabura & Stilwell 2008). 

4.2.2 Web 2.0 Technology Implementation Plan 

The majority of university libraries did not have a plan to implement Web 2.0 technology into ILI (53.49%). However, 
the number of those who did have a plan was not so different (46.51%). In respect to the latter group, most planned to 
implement Web 2.0 technology into ILI within more than a year. Notably, all who had a plan were interested in 
utilizing Social Networking. This might be because of its most familiarity (Kent 2008), its popularity among students 
(Godwin 2007, p. 108), and its promising capability to enhance all five IL skills in ACRL’s standard (Bobish 201 1). 

4.2.3 Factors Important to the Decisions on the Implementation of Web 2.0 Technology 

According to the 5-Likert scale (most important, very important, quite important, not very important, least important), 
university libraries considered both internal factors and external factors as very important. Similarly, all sub-factors 
under each type of factor were rated as very important. Table 3 shows the factors important to the decisions on 
implementing Web 2.0 technology for ILI together with the sub-factors receiving the first three highest mean scores. 

Table 3. Factors Important to the Decisions on Implementing Web 2.0 Technology for ILI* 


Factors Mean 

Internal Factors 4.12 

Policy of the library administrators 4.33 

Instructor’s (Instructors’) knowledge and skills to implement Web 2.0 technology into ILI 4.19 

Adequacy of library staff 4.14 

External Factors 4.04 

University readiness of computer facilities and Internet 4.21 

Support and co-operation of stakeholders in the university, such as administrative board, and faculty members 4.09 

Learners’ interest and co-operation 4.09 

Learners’ knowledge and skills to use Web 2.0 technology 4.05 


Under each type of factor, only the sub-factors receiving the first three highest mean scores were presented. 


Considering the internal factors, the policy of the library administrators was rated with the highest mean score. This 
might be because their policy on the implementation of Web 2.0 technology for ILI could guide and encourage the 
staff’s movement. Also, it could assure every kind of their support (i.e., money, time, resources). Analogously, the 
importance of this sub-factor could be illustrated with a survey on teaching and fostering IL programmes where lack of 
policy was a barrier (Baro & Keboh 2012). 

Instructor’s (instructors’) knowledge and skills to implement Web 2.0 technology into ILI were highly important 
since instructors should develop a deeper understanding of the technology, skills, and new ways of working (Godwin, 
2007, p. 104), so they could play with the technology to create something new and unique possible to the educational 
purposes (Magnuson 2013, p. 245). It is also critical when recognizing that lack of knowledge and skills on Web 2.0 
technology was the problem that library staff in Thai state university libraries encountered with the highest mean score 
(Paphatsurichote 201 1). 

In case of external factors, university readiness of computer facilities and Internet was the most important as it 
received the highest mean score. This might be because it is impossible to implement Web 2.0 technology for ILI 
without computer facilities and Internet. Some findings of a study on highly cited factors contributing in the successful 
application of Library 2.0, which was underpinned by Web 2.0 technology, could also confirm this (Esse 2013). 

With respect to support and co-operation of stakeholders in the university, such as administrative board, and faculty 
members, their support and co-operation could assist in a successful story of ILI (Detlor et al. 201 1, p. 578). In addition, 
since low percentage was found for the cooperation between librarians and faculty members in a study conducted on 
ILI in Thai higher education (Tuamsuk 2013), this might clarified why university libraries determined support and co- 
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operation of stakeholders in the university as the second most important external sub-factor to the implementation of 
Web 2.0 technology for ILI. 

Concerning learners’ interest and co-operation, it could definitely affect librarians’ effort to implement Web 2.0 
technology for ILI. If learners are not interested in ILI implementing Web 2.0 technology, they might not seriously 
participate in the teaching and learning activities that could lead them to essential IL learning outcomes. This is even 
more interesting when noting the findings in the Thai context that students were not interested in IL promotion activities 
(Khamhomkun 201 1). Additionally, a special attention should be taken for this sub-factor since students might have the 
conception of Web 2.0 technology for social and entertainment purposes only and they might not be aware of or 
interested in its educational potential (Luo 2010, pp. 37, 39). 


5. CONCLUSION 

Although Web 2.0 technology and IL has just become a salient topic since 2006 (Godwin 2009, p. 267), the opportunity 
Web 2.0 technology brings to ILI has drawn much attention in library domain, especially the academic ones. Based on 
the research findings, the following recommendations for implementing Web 2.0 technology for ILI are proposed: 

Since the majority of university libraries have not yet implemented Web 2.0 technology despite its advantages to 
ILI, the incorporation of Web 2.0 technology into ILI should be more reinforced. University administrators, including 
library administrators, should acknowledge the importance of ILI, particularly with the use of Web 2.0 tools, and should 
assist in the supportive atmosphere across the institution (e.g. adequate number of skilled ILI staff, equipped computer 
facilities and Internet, and co-operation among IL instructors, faculty members, and learners). Specifically, library 
administrators should encourage the implementation of Web 2.0 technology for ILI with a clear policy so that it could 
rigorously push the ILI staffs action. Apart from a proper personnel plan and job description, library administrators 
should also promote IL instructors’ knowledge and skills to implement Web 2.0 into ILI through some staff 
development activities. This is also emphasized in some literature since training on various Web 2.0 applications in 
libraries was found important (Esse 2013) and needed by the majority of librarians ( Arif & Mahmood 2012). 

Pertaining to faculty members and learners, they should mutually co-operate with IL instructors in implementing 
Web 2.0 technology for ILI. This is because positive relationship with faculty members could leverage ILI support 
(Detlor et al. 201 1, p. 578), including the experience with Web 2.0 technology. In regard to learners, they should widen 
their conception of Web 2.0 technology from social aspect and entertainment to academic practice. Moreover, they 
should acquire sufficient knowledge and skills to be able to use Web 2.0 technology effectively, safely, and morally for 
their learning. 

Last but not least, IL instructors should enthusiastically develop their knowledge and skills to implement Web 2.0 
technology into ILI. They should also experiment and create new ideas to figure out the best possibilities of integrating 
Web 2.0 technology into their instruction. They should adopt Web 2.0 technology to the most benefits it offers to teach 
all IL skills, increasingly Standard 3: Evaluates information and its sources critically and incorporates selected 
information into his or her knowledge base and value system, since least university libraries instructed it with the 
application of Web 2.0 technology. In addition, they should include various types of Web 2.0 technology in their ILI as a 
combination of Web 2.0 technology could result in a wide array of IL outcomes (Magnuson 2013, p. 250). Importantly, 
their instruction should be more active so that more desirable learning outcomes could be generated and it could have 
such an impact and sustaining influence that one active ILI session might be enough (Detlor et al. 2012, p. 156). This 
suggests that active ILI could also relieve the problem of time and staff limitation. Moreover, they should build a strong 
co-operation with faculty members as it is a key to success. They should also cultivate in learners an understanding of 
IL and its importance, especially achieving these skills with ILI implementing Web 2.0 technology. In case of the 
learners who have insufficient knowledge and skills to use Web 2.0 technology, they should provide them with some 
training. Finally, they should study the risks that might occur in using Web 2.0 technology so that they, or with the 
consultation of some experts or technologists, could overcome the problems and implement it into ILI properly. 

This study should fill some parts of a research gap. In spite of this, additional knowledge on the implementation of 
Web 2.0 technology for ILI is still required so that better understanding and application could be revealed. Since the 
current research was on the quantitative paradigm, the investigation of implementing Web 2.0 technology for ILI 
should be further carried out on the basis of other paradigms for greater depth, namely, qualitative or mixed methods. 
Also, the current research explored the implementation of Web 2.0 technology for ILI only on the side of instructors in 
libraries. Therefore, future research focusing on learners, such as their perception and achievement (Luo 2010, p. 39) as 
well as problems and needs, in regard to the implementation of Web 2.0 technology into ILI is suggested. 
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ABSTRACT 

We report on our development of an educational social machine based on the concept that feedback in communities is an 
effective means to support the development of communities of learning and practice. Key challenges faced by this work 
are how best to support educational and social interactions, how to deliver personalised tuition, and how to enable 
effective feedback, all in a way which is potentially scalable to thousands of users. A case study is described involving 
one to one and group music lessons in an on-campus, face to face, higher education context that were observed and 
analysed in terms of the actions carried out by the participants. The actions are described and it is shown how they can be 
formalised into a flowchart which represents the social interactions and activities within a lesson. Through this analysis, 
specific scenarios emerged where the feedback being given might not be effective, e.g. the recipient not understanding 
the feedback or the provision of feedback which is not specific enough. In answer to these scenarios of ineffective 
feedback, the requirements for a technological intervention which aims to make the feedback more effective are 
proposed. With this in mind, we are then able to describe a novel technological platform which has been developed as 
part of a large-scale European research project and which aims to support effective feedback. The platform is based 
around focused discussion of time based media, embedded within existing teaching activities at a research led higher 
education institution in the UK. We outline how it is being used in a blended learning model to support the teaching and 
learning of music. We reflect on the experience of developing techniques and systems for enabling communities of e- 
learning and describe our evaluation methodology which involves several, ongoing case studies and approximately 400 
users in its current phase. 


KEYWORDS 

Social learning, social timeline, feedback, MOOC, music education 


1. INTRODUCTION 

Our research project is concerned with the development of a social machine which aims to support and 
enhance the experience of learning music through the optimal provision of feedback. Key challenges we face 
in this work are how to support educational and social interactions, how to deliver personalised music tuition, 
and how to enable effective feedback, all in a way which is potentially scalable to thousands of users. 

In this paper, we present our method for addressing these challenges through an initial period of teaching 
observation and analysis followed by the development of a technological platform via a participatory design 
process. The methodology is summarised in figure 1. Following that, two key research outputs are presented: 
an analysis of one to one and group music tuition within our institution and a novel e-learning platform we 
have developed in response to this analysis. The teaching analysis resulted in a list of archetypical teaching 
and learning activities, shown in table 1, an ontology of musical feedback, shown in figure 3 and flowcharts 
describing interactions within lessons as shown in figure 2. The technological platform is essentially a 
repository for audio and video recordings which allows the user to upload media then to share it with 
communities of other users who can then place comments relating to the media along a timeline. It is 
described as a set of system requirements in table 2 and as screenshots of its media discussion interface and 
social timeline in figures 4 and 5. 
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1.1 Background 

Let us first consider what we mean by a social machine. Tim Berners Lee is credited with having coined the 
term social machine in 2000: 

Computers can help if we use them to create abstract social machines on the Web: processes in 

which the people do the creative work and the machine does the administration (Berners-Lee et al., 

2000 ). 

This quote is contextualised in the transition to web 2.0 where the process of publishing content and 
interacting online was democratised with technologies such as blogs, social networks and so on. In 2013, we 
find ourselves in the age of the social machine, where the point of interest for internet technologies is no 
longer the architectural underpinnings but the way in which people and machines interact within these 
systems. De Roure et ah, are concerned with the observation of these social machines and provide some 
examples: Wikipedia, Ushahidi, Galaxy Zoo, reCAPTCHA and Mechanical Turk (Roure et ah, 2013). 
Moving to the educational context, 2012 was the ‘year of the MOOC’ (Pappano, 2012); indeed, one of the 
authors of this paper ran a MOOC with an enrolled student body of 97,000. With their extreme student to 
staff ratios, MOOCs rely upon interactions between peers for support and assessment; this is a level of social 
interaction that seems beyond what has been seen previously within standard VLEs. Since they are 
technological systems supporting a range of social interactions, we consider them to be another example of a 
social machine. 

Now let us consider the term ‘feedback’. We define feedback in the educational context simply as a 
reaction to a learner’s output which is somehow made visible to the learner. In higher education in general, 
feedback is considered very important. It is one of the key areas covered by the UK National Student Survey 
and historically one of the lower scoring areas in terms of student satisfaction (HEFCE, 201 1). So feedback is 
important and is not always being done well, but how can we do it better? Juwah et al. present a list of 7 
principles of good feedback in higher education, wherein good feedback a) Facilitates assessment (reflection) 
in learning b) Encourages teacher and peer dialogue around learning c) Helps clarify what good performance 
is (goals, criteria, expected standards) d) Provides opportunities to close the gap between current and desired 
performance, e) Delivers high quality information to students about their learning f) Encourages positive 
motivational beliefs and self-esteem and g) Provides information to teachers that can be used to help shape 
the teaching (Juwah et al., 2004). These are useful general principles but music education is a specific case 
where the contexts and nature of feedback are perhaps quite different. Therefore, in this paper we will present 
our analysis of feedback within music education with specific examples, then show how we have developed a 
technological system which aims to support that specialised kind of feedback. 

1.2 Previous Work 

In this section, we will provide a brief overview of some related work in the areas of social discussion of 
media, online music education and peer interactions. The platform provides a media repository and timeline 
based discussion functionality; a similar commercial platform is Soundcloud, which allows users to maintain 
and share a repository of audio files and to post comments to a timeline (Bird, 2014). Considering the 
concept of annotations placed on a timeline, Latulipe discusses various projects using timeline based 
discussion systems including the ‘Video Collaboratory’ (Latulipe, 2013). Puig et al. developed the ‘Lignes de 
Temps’ software which provides a multitrack timeline aiming to promote polemical discussion (Puig and 
Monnin, 2006). Moving to the music education area, there are a range of commercial online platforms such 
as ArtistWorks (Marshall et al., 2014) and Berkley Online from the Berklee School of Music. Indeed Berklee 
have been running musical MOOCs on the coursera platform, using SoundCloud for peer discussion 
(Nuernberg and Perrier, 2013). There has also been significant public research undertaken into technology for 
music education, such as the European funded i-maestro and VEMUS projects, both of which focused in part 
on the specificity of feedback (Ong et al., 2006), (Fober et al., 2007). The concept of social interactions 
between students within VLEs did not arrive with the xMOOC in 2012, of course; the cMOOC which came 
before it had perhaps a more radical, distributed pedagogy (Smith and Eng, 2013). Going further back, 
forums have been a standard component in VLEs for a long time and new types of VLEs emphasising social 
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interactions have been reported in the literature. For example, Shi et al. describe their Topolor system which 
enables ‘social personalized adaptive e-learning’ (Shi et al., 2013). Finally, to contextualise our methodology, 
we use a grounded theory approach to analyse our lesson observations and a participatory design approach to 
develop the features of the platform (Charmaz, 2006), (Muller and Kuhn, 1993). 

1.3 Research Questions 

Our research project has several high level research questions: 

(1) How well does our approach increase participation in musical learning activity? 

(2) How important is giving and receiving feedback online for engagement with practice? 

(3) How do we correlate engagement and feedback in a community? 

(4) What is the right level of social coordination and structure that students want for online -supported 
learning? Can we provide interfaces for non-technical people to design social coordination? 

(5) How can we evidence musical competencies and musical development in students? 

(6) How can automatic techniques be used to evidence feedback in music learning? 

In the work presented here, we describe our ‘approach’ and provide evidence about the nature and 
importance of feedback which underpins several of the questions above. We also provide answers to how one 
might evidence musical competencies. 

1.4 Structure of this Paper 

The background and motivation for the work has been presented in this section. In section 2 we will describe 
our methodology for building social machines combining teaching observation and participatory design. In 
section 3 we present the outputs of the methodology including the observed teaching and learning activities, 
types of feedback and a description of the features of our new platform. In section 4 we describe the ongoing 
evaluation of the platform with 400 users. The paper ends with a discussion and conclusion in section 5. 


2. A METHODOLOGY FOR BUILDING SOCIAL MACHINES 

COMBINING TEACHING OBSERVATION AND PARTICIPATORY 
DESIGN 

The development of our platform has taken place in 4 phases. In phase 1, teaching observation, we observed 
and recorded 23 undergraduate instrumental and vocal performance lessons at our institution. The lessons 
involved 9 teachers teaching guitar, voice, piano and group and 14 individual students. The lessons were in 
either one to one or group format and spanned the popular and classical music degrees. Recordings of the 
lessons were transcribed to approximately 500 pages of text and notes were taken by the researcher observing 
the lessons. In phase 2, analysis, a grounded theory approach was used to code the activities within the 
lessons in order to identify key teaching and learning activities. This approach ‘fosters seeing your data in 
fresh ways and exploring your ideas about the data through early analytic writing’ (Charmaz, 2006). The 
activities were then organised into higher level descriptions in the form of flowcharts describing different 
types of lessons. A particular emphasis was placed on the flow of feedback between participants in these 
lesson archetypes. In phase 3, basic requirements, we drew up some basic requirements for the platform in 
order for it to support the teaching effectively. This would allow us to bootstrap the basic functionality of the 
platform ready for the next phase. In that phase, participatory design, we used a participatory design 
approach, where the input of users is sought and acted upon throughout the iterated development lifecycle 
(Muller and Kuhn, 1993). In a sense, the final phase includes its own observation, analysis and requirements 
phases, except that the observations are of users using the system (for real teaching and learning). This final 
phase is ongoing. Figure 1 illustrates the relationship between the 4 phases. 
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Figure 1 . The 4 phases of platform development 


3. OUTPUTS FROM THE METHOD 

In this section we will present the outputs generated by the teaching analysis and participatory design 
process. 

3.1 Enumerating Teaching and Learning Activities 


We were able to identify 9 distinct teaching and learning activities from our lesson transcripts and 
observation notes and these are shown in table 1 with examples of each from the transcripts. 

Table 1. The 9 distinct teaching and learning activities in one to one and group music lessons 


Activity 

Transmission 


Performance modelling 


Identify and solve 


Practicing solutions 


Feeding back 


Checking student 
understanding 


Discussion of goals and 
ideas 


Description 

Tutors provide theoretical 
and practical information to 
students 

Tutors or students perform 
good and bad examples of 
extracts from a composition 1 
Identify, discuss then suggest 
solutions to performance 
problems. 


Students put the solutions 
from the identify and solve 
activity into practice in their 
playing 

Self, peer and tutor feedback 
on a performance, after it has 
happened 

Initiated by student or tutor, 
student understanding is 
verified through dialogue 


Discussion and negotiation of 
assessment or other goals and 


Example 

so whatever you do to your mouth, it's the same sound 
because the tongue is going right up against the soft 
palette, so the sound can only come out in your nose. 

A musical activity 


Okay, did you hear that? The music is very uneven... 
Let's experiment a bit. Let's do it this way. I'll play the 
right hand with you the first time. I am going to go for 
just a legato version. Then you will have a go at it hands 
together and I would like you try to a legato version so 
then you are not affected by the separation of the notes. 
A musical activity 


That's fine, that sounded pretty good. The very first time 
it sounded - your down beat sounded a little bit like 'oh 
this is a down beat. I'm going to play loud now.' Always 
be careful about how you're shaping it. 

[Tutor ] From there, just flatten the 3 and you've got 
Dorian and add to that flatten the 6, you've got Aeolian, if 
you want to continue, what would you do next? Anybody 
know? [Student] Flatten the 2? [Tutor] Exactly right! 
Flatten the second, becomes? [Student] Phrygian. [Tutor] 
Phrygian, that's right! Which is a very nice scale. I'm fond 
of it. 

[Teacher] What is romantic for you? Let's engage in this 
kind of discussion. What is romantic? It's important. 
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Performing 

Directing 


creative ideas 


Students performing a 
prepared piece 
Tutors verbally guide a 
student performance in real 
time 


What is romantic for you? [Students]: To express your 
emotions, along with that establishing a connection. 
[Tutor] Don't you think that being romantic also 
sometimes can mean trying to be a bit more individual 
than you normally are in the real world, to be more 
special? 

A musical activity 

[Teacher] Top string this time. Take that off so you're 
playing - you want that note. There's G. Put your little 
finger back. G7. Put your finger back. The difference 
where your first finger is, yes, that's suspended, that's G. 
You can hear it. 


3.2 Teaching Workflows 


Through our lesson analysis, we were able to identify lesson archetypes which appeared several times in the 
observations. We call these archetypes ‘teaching patterns’, after Eckstein et al. (Eckstein and Bergin, 2002). 
A complete description of the teaching patterns is beyond the scope of this paper but a single example 
flowchart representing a lesson where a student performs in front of their tutor and peers can be seen in 
Figure 2. 



Figure 2. A flowchart describing a peer feedback lesson where a student performs in front of their peers and tutor then 

receives feedback 


3.3 Feeding Back about Music 

Perhaps unlike some other subjects, there is a rather discrete and finite ontology underlying the types of 
feedback one might receive about playing a musical instrument. As part of our analysis, and based on 
previous work, we have developed a detailed ontology to describe feedback on musical performance, shown 
in figure 3. It should be noted that we have identified two broad types of feedback: firstly, feedback 
connected to desirable traits in a musical performance, as shown in the majority of figure 3 and secondly, 
‘information for guiding tactics and strategies that process the domain specific information’ after Butler and 
Winne (Butler and Winne, 1995). The latter might also be expressed as encouraging the learner to develop 
their self reflective skills, their inner teacher. 
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Understanding the difference between 
one's performance and the perfor- 
mance model aimed at 

Identifying the various compo- 
nents (technical, expressive, 
stylistic etc.) that are creating a 
gap between one's perfor- 
mance and the performance 
model aimed for. 

Understanding the learning 
strategies that can help one 
to close the gap 

Understanding how one can apply 
learning strategies effectively. 
Reproducing or interpreting a 
performance model in a 
critical and/or original way 
Understanding why one's 
reproduction/interpretation of 
a model sounds/feels differ- 
ent from ones previous 
performance. 

Internalising the model 


Encouraging the 'inner teacher' Technique 


Feedback 


Movement 


Ease 

Independenc e 

Muscle co-ordination and 

Dexterity 

Strength 

Overall posture 


or support 


Note (intonation, clarity) 
Fingering (for 
instrumentalists) 

Diction (for singers) 

Articulation (legato/stoccato/ etc) 
Accuracy Expressive markings (score 
related: dynamics/timbre, 
tempo changes) 

Rhythm 

Pulse 

Metre 

Timbre control 
Sound “ ; , 

Dynamic range / control 


stage presence and stage persona. 

Physical response to the 
music performed 

Effectiveness of musical/artistic 

choices [see expressiveness & Presentation 

stylistic awareness] 

Communication with the 
audience (eg. eye contact 
when facing the audience). 


Timbre 
Dynamic range 
Textures 
Articulation 


Stylistic awareness 
(choice/control of...) 


Tempo changes 


Expressiveness / cre- 
ativity (linked to formal / 
harmonic features) 


choice of timbre/textures 
Shaping of music phrases and 
sections (groups of phrases) through 
timbral/dynamic changes & sense of 
Sound direction 

Balance of musical parts (in 
polyphonic instruments and/ 
or ensemble music) 


Musical cohesion 


Tempo 


Choice of speed 

Drive 

Groove 

Sense of breathing 
Flexibility (e.g. accelerando, 
ritardando/rallentando and similar) 
Tenuto and similar 


Figure 3. The ontology we developed which describes feedback about musical instrument playing 

3.4 An Understanding of Problems with Feedback Provision Motivating 
Essential Platform Requirements 

We now have a clear idea of the context within which feedback is given (e.g. lesson flowchart in figure 2) 
and the expected content of that feedback (i.e. the ontology in figure 3). However, we were able to identify 
several reasons why feedback might not be effective, listed below. Note that at this point, we begin to 
consider the basic requirements for our platform which will allow it to address these problems directly. 

(1) The underlying ontology driving the feedback is not well understood. The platform should be able to 
gradually expose an ontology in a range of ways. (e.g. through suggestion of relevant terms, and the 
provision of automated, high level annotations) 

(2) The feedback is not remembered. The platform should make feedback easily accessible for later 
reflection, not hidden away in a forum somewhere, for example. 

(3) The tutor is the sole source of trusted feedback. The platform should embody a community of learners 
pedagogy, to emphasise the value of feedback from peers and tutors alike. 

(4) The feedback given to peers is not honest, e.g. ‘too nice’. By building a platform that enables more 
precise feedback related to a specific ontology, feedback should naturally become more honest, as the 
emphasis for the feedback is aimed away from the individual and towards particular aspects of a 
performance. 

(5) The relevance of feedback to a particular performance is not understood. The platform should encourage 
the provision of feedback which is specific and well justified. 

(6) The feedback is too narrow. Here, the feedback focuses on a limited part of the ontology, typically due to 
time constraints in a lesson. The platform should encourage a community discussion around a greater number 
of performance aspects. 


244 


International Conference e-Learning 2014 


3.5 Platform Design 

The final phase of our methodology was the iterated development of the platform. This process is ongoing, 
but it moved through 8 versions during the first year, where increasing numbers of users were involved at 
each stage. The resulting platform is essentially a repository for audio and video recordings which allows the 
user to upload media then to share it with communities of other users who can then leave comments along a 
timeline. Its key features are listed in table 2 and shown in figures 4 and 5. At the end of this first year of 
development, the system was in active use within 5 undergraduate modules at 2 institutions. In the following 
passage, the key features and motivations for their inclusion will be discussed. 

M M jsiC Tracks »[!■ Lead sheets Profile Communities Users Help Feedback Logout 

Circle 


Social Machines Band First Jam G 



Figure 4. The music circle media discussion interface. 1) The waveform display, showing a highlighted region, 2) The 
tagging dialogue, showing a drop down list of pre-used tags 3) The social timeline, showing sets of time linked comments 
created by several users 4) a discussion thread based on a single region in the recording, including an embedded youtube 

video 
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Table 2. The key features of the system compared to some pre-existing systems which we have used for teaching at our 

institution 


Feature 

MusicCircle 

SoundCloud 

Mahara 

Easy access, personal media repository with mobile media 
capture client 

X 

X 


Simple, transparent sharing 
and community model 

X 


X 

Intuitive discussion interface with content prompting 

X 

X 


Social timeline with region selection 

X 



Powerful discussion system 

X 


X 

Automatic feedback agent 

X 



Suitability for use as a research platform (data access, privacy 
etc.) 

X 


X 


Easy access, personal media repository with mobile media capture client 

The aim is to remove barriers to content uploading and sharing and to make content easily accessible for 
later review. The platform includes simple record and upload apps for iOS and Android to make content 
addition as easy as possible as we identified that the often over-complex process of putting content into VLEs 
can be a serious barrier to uptake for students. 

Simple, transparent sharing and community model 

The aim is to increase user confidence in uploading and sharing media. The platform provides a very 
clear method of controlling who the content is shared with. Also, users can delete any comments made about 
their content. 

Intuitive discussion interface with content prompting 

This feature aims to motivate commenting activity and to encourage use and understanding of appropriate 
terms from the ontology. 

Social timeline with region selection 

Feedback is always connected to a particular range of time in the media. Also, all commenting users have 
individual timelines displayed below the media. This promotes awareness of the community opinions, 
making feedback specific to a person and a time. 

Powerful discussion system 

Users can reply with audio, video, text and so on. Audio and video responses within the platform can then 
become a subject for discussion in themselves, with their own social timeline. 

Automatic feedback agent 

We are developing software agents which are able to feed- back automatically about musical 
performances. They work by comparing different perfor- mances and making high level comments about the 
variations, connected to the feedback ontology. This provides a ‘neutral’ source of feedback and exposes the 
learner to the on- tology. A full description of the feedback agent is beyond the scope of this paper but it is 
built around machine learning and audio analysis techniques. 

Suitability for use as a research platform (data access, privacy etc.) 

We need to be able to ensure the data is appropriately protected and that it can be accessed for analysis 
throughout the project. Also, we need to be able to rapidly prototype and integrate different components to 
our platform for experimentation. 
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© Comment ^ Tag |j Attach file to track Share with community 


00:02:87 



anonymous, 139 days ago 0edit 1C delete 

Irregular pulse. 


C5 reply 


Figure 5. The social timeline, showing sets of annotations from two users. Each block in the timelines represents an 
annotation connected to a specific region in the recording. Here, the top timeline was created automatically by the 

feedback agent 

The key features of the platform are listed in table 2, where we also compare them to the closest 
equivalent commercial system, SoundCloud and a well known open source e-learning tool with social 
features, Mahara. 


4. A DESCRIPTION OF OUR ONGOING EVALUATION WITH 400 
USERS ACROSS 2 INSTITUTIONS 

The participatory design process aims to suggest then optimise platform features. In a sense, this represents 
an ongoing, evaluation and improvement cycle. However, as stated in the introduction we are interested in 
the evaluation of social machines and the activities they enable at a higher level than basic platform features. 
In this regard, we are running significant case studies with our platform with approximately 400 users spread 
across 2 institutions and 5 different modules. The evaluation scheme consists of qualitative and quantitative 
methods. In particular, we will be using interviews, survey tools and user activity metrics including social 
network analysis. This will allow us to address the research questions listed in section 1.3 with a variety of 
perspectives. We anticipate being able to analyse a data set containing hundreds of media items, thousands of 
comments and many thousands of interactions. 


5. CONCLUSION 

Work has been presented which faces the challenges of how to support educational, social interactions, how 
to deliver personalised music tuition, and how to enable effective feedback. A methodology for addressing 
the challenges has been described which takes real observational data and analyses it into formalisations of 
teaching and learning activities. The outputs from this methodology have been presented, including a list of 
key teaching and learning activities, a flowchart describing the interactions within a typical lesson, and an 
ontology of types of feedback. It has been shown how the outputs have been iteratively interpreted into the 
design for a novel e-learning platform driven by social interactions and effective feedback. The current 
system has been introduced and the ongoing evaluation with 400 users has been described. The immediate 
targets for our future work are to increase the number of learners operating within the platform, to conduct an 
investigation of the wider applicability of the system, for example as a means to deliver recordings of lectures 
and the development of our tool kit for quantitative evaluation of the system. Inspired by the examination of 
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the importance of feedback presented here, the longer term goal is to develop a deeper understanding of the 
nature and importance of feedback in the learning and creative processes. 
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ABSTRACT 

For supporting the education of debugging skills, we propose a system for generating error correction questions of 
programs and checking the correctness. The system generates HTML files for answering questions and CGI programs for 
checking answers. Learners read and answer questions on Web browsers. For management of error injection, we have 
analyzed types of errors and defined the processes of error injection as code transformation patterns. The system 
synthesizes code fragments including errors by transforming correct code fragments according to the selected patterns. 
Full coverage of all possible answers is difficult. Instead, we have adopted a strategy to restrict editable points and 
possible answers from the educational view. To confirm the effectiveness of the system, we have generated questions 
using several examples and applied them to a programming exercise as an evaluation experiment. 


KEYWORDS 

Error Correction Questions, Programming Education, Debugging 


1. INTRODUCTION 

In programming courses, students learn skills for coding, code reading, and debugging through various kinds 
of exercises. Although debugging is important for actual software development, it is difficult for students, 
even who have a good understanding of programming, to acquire skills to debug programs effectively [6][7]. 
In this paper, we focus on learning support for debugging, whose controlled exercises are difficult to be 
provided. 

Typical exercises in programming courses are describing full codes and filling empty boxes embedded in 
the texts of codes. Through describing full codes, learners find errors and inevitably try to debug for 
detecting faults. Debugging experiences, however, are different for each learner, and teachers cannot control 
experiences for them to use all necessary debugging skills. In the case of exercises using questions for filling 
empty boxes, learners may not have chances to debug because the codes they read do not include any errors, 
and they can fill answers without compiling and testing. For controlling their experiences of debugging, 
exercises using error correction questions are suitable. Learners read codes and need to detect errors that are 
injected purposely. Teachers can select errors for injecting according to debugging skills to be learned. 

Preparing error correction questions is, however, a tough work for teachers. It requires injecting various 
errors in code fragments and check the correctness of all possible answers, some of which may be unexpected 
one but correct. There are some researches for supporting these work flows [2][7][8][10], but a few ones for 
automatic systems of question generation and correctness checking [2][8]. The key questions about 
constructing the systems are how to manage error injection into code fragments and how to cover all possible 
answers in correctness checking. 

In this paper, we propose a system for generating error correction questions and checking the correctness. 
The system generates HTML files including editable text forms for answering questions and CGI programs 
for checking answers. They are deployed on a Web server and learners read and answer questions on Web 
browsers. For management of error injection, we have analyzed types of errors and defined the processes of 
error injection as code transformation patterns, which we call error patterns. The system synthesizes code 
fragments including errors by transforming correct code fragments according to selected error patterns. If the 
system allows the learners to edit any points in codes, it needs to accept all possible correct answers. Full 
coverage of all possible answers is difficult. Instead, we adopt a strategy to restrict possible answers to the 
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ones that is reasonable for learning objectives, which mean the syntax and semantics of the language, typical 
program descriptions, and algorithms that learners should understand. We have analyzed types of possible 
answers and propose constraints of editable points in codes on answering questions. For evaluating the 
system, we have implemented a prototype of the system and generated questions for learning the syntax and 
semantics of conditional branches, loops, arrays, strings, pointers, and structures, and a sorting algorithm. 

The main contribution of this paper is to provide a fundamental framework of automatic generation and 
checking answers of error correction questions for debugging exercises. On the framework, the errors are 
defined as code transformation patterns, and this makes it easy for teachers to add new programs and errors. 
We hope that the discussion about what errors and programs are effective in terms of acquiring debugging 
skills and how we collect them in practical programming exercises is opened up. 

In the following, we analyze error correction questions and discuss reasonable constraints in Section 2, 
and show a design of the system in Section 3. In Section 4, we show an experiment for evaluating the system 
and discuss its results. We show related works in Section 5 and conclude our research in Section 6. Though 
we use the C language in this paper, our study is not restricted to it. 


2. TYPES OF ERROR CORRECTION QUESTIONS 


2.1 Error Correction Questions 


1 

iinclude <stdio.h> 


1 

iinclude <stdio.h> 

2 

gdefine MAXSIZE 128 


2 

idefine MAXSIZE 128 

3 



3 


4 

int main {void) 


4 

int main (void) 

5 

( 


5 

( 

6 

int data [MAXSIZE] ; 


6 

int data [MAXSIZE] ; 

7 

int size, sum, count, i; 


7 

int size, sum, count, i; 

8 

double avg; 


8 

double avg; 

9 



9 


10 

sum = 0; 


10 

sum = 0; 

11 

size = 0; 


11 

aize = 0; 

12 

while (scanf ("^d”, (data [size]) 


12 

while (scan("%d~ . _data(size]) 

13 

! = EOF) { 


13 

i= EOF) { 

14 

sum += data [size]; 


14 

sum += data [size]; 

15 

size++ ; 


15 

size++; 

16 

) 


16 

) 

17 



17 


18 

avg = (double) sum / size; 


18 

avg « (double) sum / size; 

19 

printf(“avg = %f\n", avg) ; 


19 

prjn frf (“avg = %f\n", _avg) ; 

20 



20 


21 

count = 0; 


21 

for (i =1; i £ size; i++) { 

22 

for (i = 0; i < size; i++) { 


22 

if (data [i] > avg) { 

23 

if {data[i] > avg) { 


23 

count++; 

24 

count++ ; 


24 

} 

25 

) 


25 

> 

26 

) 


26 

printf ("area ter than avu: %d\n" . 

27 

printf ("greater than avg: %d\n". 


27 

_count) ; 

28 

count) ; 


28 


29 



29 

return 0; 

30 

return 0; 


30 

) 

31 

) 





(a) Correct Program 


(b) Program including Errors 


Figure 1. Error correction question (1): condition, loop, and array 
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An error correction question contains a program code including errors and requires the learners to modify 
them correctly. An example is Fig. 1, whose specification is "Calculate the average value while reading a list 
of integer values in a file, and then print a number of values that are greater than the average." Program (a) in 
Fig. 1 is a correct program, and program (b) includes four errors: in Line 12, (1) the function name scan is 
a misspell of scanf, and (2) an address operator & is missing in an argument; in Line 21, (3) the initializer 
and the condition in the for loop do not match to the range of the array referred in the loop; in Line 23, (4) 
variable count is referred without initialized. The underlined italic texts are editable elements, that is, 
learners can modify them. The blank lines at Line 9, 17, 20, and 28 are also editable, where learners can add 
statements. When a question has many editable elements, it tends to become difficult and to allow 
unexpected answers. In Section 2.5, we discuss the restriction of editable code elements from the educational 
view. Fig. 1 is a question designed for beginners of programming who is learning the syntax and semantics of 
conditional branches, loops, and arrays. By this question, we hope that learners learn the usage of the scanf 
function, necessity of variable initialization, and a typical loop to scan an array. 


1: struct person { 

2: char nans [64]; 

3: double height, weight; 

4: ) ; 

5: 

6: void f ilnuh 1 a *d , riniih I e *b) 

7: [ 

8: double _tnp; 

9: trap ■ *a; *a ■ »b; *b ■ tmp; 

10: ) 

11 : 


12: void eort_by_height( 

13: struct person p[], lnt size) 

14: { 

15: int min, i, j; 

16: 

17: for (i “0; i < aize-1 : i++) ( 

18: min > i; 

19: for (j = i+1; j < j++) ( 

20: if tp til . hei hgfc i plininl .height ) ( 

21: min ■ j; 

22: } 

23: ) 

24: swap ( pill . height , p [mini . height 1 ; 

25: ) 

26: ) 


Figure 2. Error correction question (2): a sort program of an array of structures 

We show another example in Fig. 2. It is a more complicated program using pointers, structures, and the 
selection sorting algorithm; the structure person represents a personal physical datum for each person and 
the function sort_by_height sorts an array of the structure. The sort function gets an array by argument 
p and its size by argument size, and it sorts p in ascending order of a member height with the selection 
sort algorithm. It contains five errors: (1) in Line 6 and 8, the type of function swap's arguments and local 
variable tmp must be struct person; (2) in Line 19, the condition must be j < size; in Line 24, (3) 
all address operator &s are missing for both arguments; (4) the index j of array p must be i; (5) all member 
references of height are unnecessary. 

These examples imply that error correction questions are more difficult than fill-in-the-blank questions. 
Fig. 3 shows (a) an error correction question and (b) a fill-in-the-blank one about if-branches and strings. 
The function strconv changes upper letters in the ASCII string s to lower ones and lower letters in s to 
upper ones; for example, "Hello World! " is changed to "hELLO wORLD ! ", Program (a) includes three 
errors: in Line 3, (1) the equivalence operator in the condition of the while is wrong; (2) the logical 
operators in Line 4 and 7 are wrong; in Line 7, (3) else is missing before if. In our experiences, error (3) 
is relatively hard to find at a glance, although this program is simple and short. On an error correction 
question, firstly, learners read a program, secondly, they find errors, and finally they correct the codes, that is, 
adding, deleting, replacing, or moving codes. Finding and correcting errors require understanding the control 
flows, data flows, and logics of programs in addition to the syntax of the language. On the other hand, 
program (b) shows explicitly blank boxes where learners need to write correct codes. They may write correct 
codes without comprehension of programs. For example, at the first blank in Line 7, there are clear hints, if 
in Line 4 and the following parentheses. Furthermore, from the syntactical correctness, it should be filled 
with either if, else if, while, or switch. 
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1: void strconv(char *s) 

2 : < 

3: while (*s □’\0’) { 

4: if (■«■<= *a *s <= '*■) { 

5: *s = *s - 'a' + ’A' ; 

6 : } 

7: Q^CA' <= *sQ^|** <= ■ Z ' ) { 

8: *s = *a - 'A' + 'a' ; 

9: } 

10: a++; 

11 : ) 

12: } 


(b) fill-in-thc-blank question 

Figure 3. Error correction question and fill-in-the-blank question: if -branches and strings 

When we generate error correction questions, we need to maintain the readability of codes. Codes should 
have comprehensive structures of the units of inputs, main processes, and outputs because learners may have 
less skill for reading. Unnatural descriptions in codes may give undesirable hints to learners. We have 
adopted the following coding styles: (SI) a variable is used for one purpose; (S2) the initialization of a 
variable that is used in a loop should locate just before the loop; (S3) two semantic process units are 
separated by a blank line; (S4) no successive blank lines are allowed. While injecting errors, removing 
statements may cause two successive blank lines in codes. They should be integrated into one blank line 
because they indicate a definite lack of a statement. For example, in Fig. 1 (a), when we remove the 
assignment statement in Line 21, we also remove the following new line because Line 20 is a blank line. 


Table 1 . Samples of errors for error correction questions of programs 


Learning Objective 

Error 

Edit 

Operation 

Correctable 

Point(s) 

Possible 

Represen- 

tations) 

Type 

Reading primitive values 
with scanf function 

Missing address operator &s 
before arguments (Fig. 1 error (2)) 

delete 

one 

one 

(a) 

Reading strings with 
scanf function 

Unnecessary address operator &s 
before arguments ( char [ ] ) 

insert 

one 

one 

(a) 

Operators 

Wrong operator (Fig. 3 error (1),(2)) 

replace 

one 

one 

(a) 

Branches 

Missing else (Fig. 3 error (3)) 

delete 

one 

one 

(a) 

Loops 

Wrong initialization and condition 
in a loop (Fig. 1 error (3)) 

replace 

one 

multiple 

(b) 

Types 

Wrong type (Fig. 2 error (1)) 

replace 

one 

one 

(a) 

Structures 

Unnecessary member references 
(Fig. 2 error (5)) 

insert 

one 

one 

(a) 

Data flows of variables 
and initialization 

Using a variable without 
initialization (Fig. 1 error (4)) 

delete 

multiple 

one 

(c) 

Data flows in a loop 

Moving the assignment before a 
loop inside the loop 

move 

multiple 

one 

(c) 

The elements to be 
swapped in the selection 
sort algorithm 

Using wrong indexes on swapping 
elements in an array. 

(Fig. 2 error (4)) 

replace 

one 

one 

(a) 
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2.2 Errors for Questions 

For analyzing types of errors used in error correction questions, we have collected errors in our programming 
courses and the related papers [3][6][9][11]. These papers have reported that the students often make 
syntactical mistakes such as missing a semicolon and lacking one of a pair of curly braces or parentheses. In 
general, it is relatively easy for students to correct syntactical errors because a compiler detects them and 
shows error messages with line numbers. We have collected logical errors such as using a variable without 
initializing or initialized with a wrong value because these errors significantly contribute the development of 
debugging skills Compilers do not detect these types of errors, and learners need to read a program carefully 
and understand its logic. 

We show examples of errors and the learning objectives of them in Table 1. We have analyzed these 
errors in terms of edit operations and the numbers of correctable points and possible representation. A 
correctable point means an editable point at which we can write a possible representation. A possible 
representation is one of the code fragments that are acceptable as correct answers. It may be semantically 
different from the original (see Section 2.4). 

2.3 Edit Operations for Injecting Errors 

We have analyzed edit operations to inject errors in code fragments. We have classified the operations into 
four types: insert, delete, replace, and move. The column edit operation in Table 1 shows operations for each 
error. Though an operation move is a combination of delete and insert, we have identified it as a primitive 
because the two operations should be occurred simultaneously. 

The errors injected by deleting code elements or replacing code elements with others are the most typical 
ones. A missing assignment of an initial value and a wrong relational operator in a loop condition are 
examples. They are caused by deleting the assignment or replacing the operator with another one. Error 
correction questions of theses types are relatively easy to prepare. The number of these errors we found is 
greater than the ones of others. Instead, the errors of insert and move are restricted. Though it is not difficult 
to insert or move an element in codes, these errors would happen rarely because they break data flows or 
introduce unnecessary statements in contexts, and it is easy for learners to find them. 

If we may not consider all possible answers, checking the correctness of answers becomes easy to be 
implemented; the system needs only to check that each modified element is same with the original one. For 
the move operation, the system needs to check both of modifications of insert and delete are correct. The 
difficulty is to cover all possible answers, and we discuss it in the next section. 

2.4 The Number of Correctable Points and Possible Representation 

We have analyzed correctable points and possible code fragments for the errors that we found. Possible 
answers mean a set of answers whose positions in codes and representations are different. We ignore the 
differences of the representations of syntactical equivalent expressions, such as a difference of a+b and b+a 
because it is possible to generate all representations of expressions systematically. Our system is for 
beginners education, and the expressions of the answers are small, for which it is possible to generate all 
alternatives in practical time. We also ignore differences of white spaces. We explain how to ignore the 
differences in our implementation in Section 3.1. In the following, we consider the possible answers whose 
position and/or semantics are different. 

The results of the analysis are shown in Table 1 as the numbers of correctable points and possible 
representations. For considering the difficulty of the correctness checking, we categorize errors into four 
types: (a) one correctable point and one representation, i.e., not existing any other answer, (b) one correctable 
point and multiple possible representations, (c) multiple correctable points and one representation, and (d) 
multiple correctable points and multiple possible representations. 

Type (a) represents the errors for which the original positions and code fragments are the unique answers. 
The supporting system checks only one position and one fragment for each error and is easy to be 
implemented. 
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Type (b) represents the errors for which the original positions are unique answers, but there are variations 
in code fragments to be placed. For example, an n-times loop including an error, for (i=l; i<n; i++) 
can be corrected as either for (i=0; i<n; i++) or for (i=l; i<=n; i++). They cannot be 
checked syntactically, and this correctness depends on contexts. 

Type (c) represents the errors for which the original code fragments are the unique answers, but the 
position of each error is not fixed to one. An error of this type occurs when a statement is removed. For 
example, in Fig. 1 (b), the initialization of variable count can be inserted at any lines between Line 9 and 20. 

Type (d), which does not appear in Table 1, represents the errors that have multiple answers in positions 
and code fragments. This type of errors makes the system complicated. The answers other than the original 
ones tend to be unnatural and inadequate from the educational view. For example, we can correct the errors at 
Line 21 in Fig. 1 (b) by replacing data[i] with data[i-l] in Line 22. If we restrict the correctable 
points of the above example to Line 21, the error becomes type (a), which is the simplest type. This example 
suggests us that restriction of acceptable answers is a reasonable approach. 

2.5 Restriction of Correctable Points and Editable Points 

How to restrict correctable points is a difficult question because they depend on the semantics of errors and 
target codes. One of the simplest ways is to restrict to the original positions and code fragments, as type (a). 
The errors in type (b) can be changed to type (a) by restricting correctable points. For example, in an error 
code fragment of an n-times loop, for (i=l; i<n; i++) , by allowing modification of the operator < in 
the condition, only for (i=l; i<=n; i++) can be acceptable. For accomplishing this, the system needs 
the ability to modify fine-grained elements. 

Strong restrictions, however, provide clear hints to learners and easily lead them to the answers. To avoid 
correctable points becoming clear hints, we introduce editable points. Editable points exist at all positions 
similar to correctable points and the learners cannot distinguish them on the system. For example, in Fig. 1 
(b), the correctable point of lacking operator &s is at the front of the second argument at Line 12, and the 
editable points are at the front of the second arguments in the printf functions at Line 19 and 27. Editable 
points include correctable points, and we can define them in the same way of correctable points. The 
difference between them is the modification of texts. When they are injected in the codes, the texts at the 
correctable points are replaced with error ones, but the ones at the other editable points are not changed. This 
introduction of editable points embraces the contradiction that they increase possible answers, which have 
been excluded by the restriction of correctable points. Unfortunately, this contradiction is not able to resolve 
systematically. How to select patterns for editable points are responsible for the users as teachers. 

An error with multiple correctable points, like type (c), occurs when an assignment is deleted. The error 
can be corrected by adding the original assignment at any point unless it is to preserve the original data flows. 
Strictly speaking, the correctness checking of type (c) requires data flow analysis, which makes the system 
complicated. To keep the system simple, we introduce the constraints reasonable from the educational view: 
(i) only assignments for initialization can be deleted; (ii) editable points where learners can insert statements 
are restricted to blank lines. The rule (i) is introduced because missing initializations are typical errors, and 
the questions deleted other kinds of assignments become the same with fill-in-the-blank ones for 
understanding algorithms which are out of our targets. The rule (ii) prevents the learners from confusing by 
multiple possible answers. Under these constraints and the coding style (SI) and (S2) in Section 2.1, we can 
identify the valid correctable points without data flow analysis, which are blank lines located between sibling 
statements before the deleted assignment. For example, in Fig. 1 (b), the correctable points are the blank lines 
at Line 9, 17, and 20; the one at Line 28 is not a correctable point. 

The errors of type (d) are problematic, but they are minor in our experiences. The purpose of the error 
correction questions is to develop skills for identifying typical errors. The errors of type (d) deeply depend on 
semantics of code, and they are not typical. We should avoid this type of errors or try to change it to type (c) 
or (a) by restriction of editable points. In this paper, we do not discuss this anymore. 
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3. AUTOMATIC GENERATION OF ERROR CORRECTION QUESTIONS 


3.1 Overview of the Supporting System 

We have developed a supporting system of error correction questions. Fig. 4 shows an overview of the 
system. The system consists of two components. One is a presentation system that provides a set of error 
correction questions to the learners, and they answer on it. The answers are checked by the system, and the 
results are returned to the learners. It is implemented as a Web system, i.e., a set of HTML files, JavaScript 
programs and Perl CGI scripts that are deployed on a web server. The learners read and answer questions on 
web browsers. The other component of the system is a generating system of error correction questions. A 
user as a teacher selects a correct program and error patterns. The system applies the patterns to the correct 
program and generates both of an HTML file of the question and a CGI program for checking answers. The 
HTML file contains a program including injected errors and editable points. 



Figure 4. An overview of the supporting system for error correction questions. 

The checking CGI programs can accept variants of expressions, such as a variant b+a for a+b. The 
equivalence of expressions is complicated to evaluate in practice, and the development of the evaluation 
algorithm is out of the scope of this paper. We have implemented an evaluation code optimized for our 
examples. Answers for all examples are small expressions, and we do not accept unnatural answers, such as 
a+0 for a and 5-5 for 0. Therefore, the number of variants that we should consider is small. 

The presentation system needs to hide editable points from the learners. If all editable points explicitly 
appear on question pages, the learners can easily guess the answers. For this requirement, we have adopted 
JEIP 1 , a plug-in of jQuery, which hides editable points from a learner until the user moves the mouse over the 
texts on them. If the user clicks an editable point, an input field and a save button appear at the point. After 
changing and saving all editable points where the user consider need to modify, the user can submit answers 
to the system by pressing the submit button. 

A demonstration of error correction questions is available at http://ecq.tebasaki.jp/. 


1 jQuery Edit In Place (JEIP), http://josephscott.org/code/javascript/jquery-edit-in-place/ 
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3.2 Error Patterns 

For managing correct programs and errors separately, we define errors as code transformation patterns. We 
also define editable points in the same way of errors, which do not modify programs in practice. This 
separation makes it easy to add a code fragment and another type of errors newly. We have implemented the 
system on a program transformation, called TEBA [4], which has a parser of code fragments including 
additional symbols, such as program patterns. TEBA also provides a transformation system on token-based 
syntax trees, and it allows modifying the fine-grained elements as discussed in Section 2.5. 

An error pattern consists of two parts, before-part and after-part. A before -part is a target code to be 
modified, and an after-part is a new code as a replacement. For abstraction of syntactic elements, typed 
pattern variables can be used. In error patterns, the correct tokens in the before -part and the error tokens in 
the after-part are surrounded by the special tag, <0 and 0>. In an error patterns of delete type, the tag in 
after-part surrounds an empty token since error tokens never exist. The line where the tag exists becomes a 
blank line after the error pattern is applied. In an error pattern of insert type, the correct tokens in the before- 
part are empty. In one of move type, we use the tag, <#0 and 0>, for grouping a set of deleted tokens and 
inserted ones. We describe an injection of editable points as an error pattern whose before-part and after-part 
are the same. We show three examples of error patterns in Fig. 5. 

% before 

tor ( ${al :ZXPR) m <q a g>; ? : EXPR} <1 < «> ${a2r :EXPR} ; ? 1»J : EXPRi ) 

${s:STMT*> $; 

'6 after 

for ( fel - <e 1 e>; ${*21} <8 <« e> $ { c2r } ; 9I«3J) 

$; 

% end 


(a) replacing initializer and the operator of the condition in a for loop (Fig. 1 error (3)) 


% before 

<8 »fvar:ID FVAR) = $ {lit : LITERAL) ; 8> 
tor (Slal :EXPR) ; $(«2:EXPR}: 9{e3:EXFR) ) 
9 ( a : STMT*) 3; 

% after 

<e e> 

for (${al ) ; ${*3} ) 

${•> *; 

>. end 

(b) deleting an assignment just before 
a for statement (Fig. 1 error (4)) 


%before 

Gwap(?(al:EXPR)<e <l>, S { a.2 : EXPR ) <8 »>) : 
tatter 

3irap(9<al}<8 .height 9 > . 9(a2}<6 .height 8» ; 
tend 


(c) inserting unnecessary member references 
. height (Fig 2. error(5)) 


Figure 5. Error patterns as code transformation 


4. DISCUSSION 

4.1 Generality of Error Patterns 

To confirm that the system can generate error correction questions, we have defined 15 patterns for injecting 
errors and 10 patterns for setting editable points, and applied them to 4 sample programs: Fig. 1, Fig. 2, Fig. 
3, and a binary search function. The error patterns we defined are varied in generality. For example, a pattern 
of removing addressing operators &s can be applied to all code fragments that include the operators. The 
error pattern Fig. 5 (c) is less general, which depends on the function swap and the structure person. In our 
defined patterns, 10 of 25 patterns are program specific. These patterns are difficult to be reused, but they are 
useful for generating questions optimized for learning objectives. For example, in Fig 2., while we have 
described an error pattern for inserting unnecessary references (Fig. 5 (c)), we have selected a member name, 
height, because the specification of the program requires to sort data by the member. We have set an 
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editable point on the indexes of p [ j ] , but not the ones of p [min] . If both of them are editable, another 
answer becomes acceptable; the answers, p[min] .height > p [ j ] .height in Line 20 and 
swap (&p [min] , &p [ j] ) in Line 24, are valid. We also have made a space editable before variable tmp 
in Line 8 because we expect that learners may insert * in the same way of the parameters *a and *b without 
thinking semantics deeply. 

From our experiences, the error patterns for learning the syntax and semantics of the language tend to be 
general, and the ones for learning algorithms tend to be specific. For improving the reusability, we need to 
investigate the way to make program specific patterns more general. It may be possible to describe the 
contexts of applying patterns and its edit operations separately. Though the contexts depend on programs, we 
expect operations can be described in general styles. 

4.2 Experiment and Evaluation 

We also have made an experiment of the programming exercise using the generated error correction 
questions. The subjects are 10 undergraduate students in the third grade, who had taken programming courses 
and have the skills for developing small programs. The purpose of the experiment is to find unexpected 
answers. As a result, we have not found unexpected answers that are correct. This means the restriction of 
our approach works as expected. We, however, have found redundant answers. For example, a student has 
inserted avg = 0 ; at Line 9 in Fig. 1 (b). The variable avg is never referred before the assignment at Line 
18, and no initialization is needed. In the question of Fig. 2, there was an answer that the condition of the 
for at Line 17 was changed to i < size. Though it executes the redundant process on i == size-1, it 
produces the same result as the original program. Flow to treat redundant answers depends on the purpose of 
the exercises. Our system cannot support the case that the teachers want to judge redundant answers as 
correct because the correctness checking is implemented based on text matching. For checking redundant 
answers, it may be effective to compare the results of the original codes and the answered codes by tests for 
the sufficient coverage of inputs, while it makes the system more complicated and requires more efforts for 
teachers to prepare adequate test sets. 


5. RELATED WORKS 

AEGIS[2] is a system to generate questions from XML documents and supports three types: multiple choice, 
fill-in-the-blank, and error correction. An error correction question is generated from a multiple choice 
question by fusing the choices to the code. AEGIS requires describing a full XML description of questions, 
including codes and errors, and does not support automatic synthesis of questions. Our system separates 
descriptions of codes and error patterns and synthesizes questions from them. The interface of error 
correction questions generated by AEGIS is quite simple. It shows the text of a code on which errors are 
marked, and the learner inputs answers in the text fields. The interface of our system hides errors and allows 
modifying the elements that are not in error. Our system provides an exercise environment similar to the 
traditional ones using papers and pens. 

Itoh et al. proposed a method for generating error-correction exercises for learning algorithms [10]. It 
determines the fault positions by the algorithm design paradigm and injects faults by the syntax-directed 
faults patterns, which are specific to the algorithm education. By specifying a set of correct programs, an 
algorithm design paradigm such as divide-and-conquer, the number of errors to be injected, and the number 
of source code files to be generated, the system generates source codes including errors automatically. 
Though it supports error injection, it does not propose a method for the correctness checking. Our system 
allows injecting an error at any positions, and it does not depend on specific domains of program educations. 
However, it requires manual selection of error patterns while considering possible answers. A support 
method for selecting patterns is a future work. Patterns should be selected from the multiple views such as 
learning objectives, the difficulty of questions, the degree of learners’ understanding, and so on. 

From the view of code transformation, our system is a kind of a mutation system [ 1 ] [5], A mutation 
system generates multiple variants of a code by adding small changes to its copies. The distinctive 
application is the test set evaluation in which test sets are tested how many variants they can detect as errors. 
Mutant systems add changes randomly, but our system adds changes in a restricted manner. 
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6. CONCLUSION 

We have proposed a support system for generating error correction questions and checking answers. On the 
system, we can describe error patterns of injecting errors to codes, and the system synthesizes questions from 
them. The system also generates CGI programs for checking answers. We have collected some typical errors 
in programming and generated questions. We have showed the validity of our system by a small experiment. 
We need to make experiments with a large number of subjects. 

Collecting other errors that are effective for acquiring debugging skills is a future work. Although some 
researches have reported syntax errors by novice programmers [3][6]][9][1 1], a few logical errors are known. 
While we can collect syntax errors by logging compile errors, to collect logical ones we need to investigate 
programming processes; we have an interest in how they correct faulty programs, for which compilers report 
no error. Work-in-progress codes and input data they select to test programs may be helpful for teachers to 
analyze their errors. An integrated development environment for education is one of the suitable systems to 
preserve learners’ processes. 

We hope that discussion about learning support for debugging using error correction questions is opened 
up. 
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ABSTRACT 

This paper presents a model for conducting Internet of Things (IoT) classes based on a web-service oriented cloud 
platform. The goal of the designed model is to provide university students with knowledge about IoT concepts, 
possibilities, and business models, and allow them to develop basic system prototypes using general-purpose micro- 
devices and a cloud and service infrastructure. The model was based on a cloud infrastructure deployed at the E-Business 
Department at the Belgrade University, and some implementation details are given. The model was tested and evaluated 
in a pilot course. 


KEYWORDS 

Internet of Things, Raspberry Pi, Arduino, Cloud Computing 


1. INTRODUCTION 

The expression “Internet of Things” describes the existence of a number of various things or objects like tags, 
sensors, actuators, mobile devices, capable of cooperating in order to achieve a common goal (Atzori et al. 
2010). Such intelligent devices can take a number of forms and roles, and the composition of such systems 
can be adjusted dynamically, according to the needs of the users. This gives IoT an almost unlimited area of 
application within both business and industry (process management, intelligent transport, automation), as 
well as in homes and public environments (smart homes, e-health, assisted learning). 

The term IoT encompasses an unbounded, growing set of devices and technologies, and as the IoT 
technologies gain traction globally, the need for experts that combine knowledge from various technical 
fields increases. IoT projects are likely to need designers, system integrators, developers and technicians in 
order to take an idea from inception to execution. Such diverse requirements can create an understanding gap 
between business-oriented individuals and their ideas, and the actual implementers that deal with realistic 
constraints. Ideally, an individual with an IoT business idea would be able to understand the possibilities and 
work in a small team, developing a prototype using off-the-shelf parts. 

Introducing IoT into an environment is accomplished by introducing and interconnecting intelligent 
devices and, essentially, making an environment intelligent and supportive of any human activity. 
Applications of IoT are therefore as diverse as human activities and environments are. It is impossible to 
foresee the specifics of future IoT development, but some currently relevant, broad domains of application 
include transportation, logistics, healthcare, smart homes/offices/plants, and personal and social domains 
(Atzori et al. 2010). 

Internet of Things represents an advanced paradigm, one that requires technology, knowledge and 
infrastructure, available in rich, developed countries. However, IoT solutions and especially IoT education 
can immensely benefit developing countries, offering a way of catching up faster, as well as a profitable 
industry for outsourcing, with predictions of IoT market being worth 22 to 50 billion dollars in 2020 
(Schlautmann et al. 201 1). Thanks to the cheap micro-devices like Raspberry Pi and Arduino, it is possible to 
develop various systems with less investment into infrastructure. The Raspberry Pi is of special interest, since 
it represents an entire computer the size of a credit card, and some systems based around it can be found in 
(Raihan 2013)(Kaloxylos et al. 2014). Raspberry Pi and similar devices are generally complemented by 
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software APIs that abstract low-level operations, allowing effective utilization from a higher perspective, 
which is well-suited for individuals with a background in business informatics. 

Traditional teaching in practical engineering areas usually has a twofold structure, where the first part 
presents theoretical foundations, and the second introduces real-world issues and applications. A different 
approach explored by some institutions gives more freedom to students, allowing them to choose the 
direction, breadth, and depth of their education, as well as combine their pre -knowledge from other areas of 
study (Director et al. 1995). These stipulations can still potentially be applied even in the context of a single 
IoT course, especially if it is not strictly hardware -oriented. Small classes comprised of business informatics 
students could produce a motivating and individual experience for every student by taking advantage of their 
diverse background. 

Inclusion of technology into higher education is well suited for models based on constructivism and 
socialization, and can transform the educational process by making it more effective and attractive to students 
(Bustos Andreu & Nussbaum 2009). IoT classes can capitalize on this effect since they inherently deal with 
technological gadgets and information communication technologies. Several types of environments also 
attempt to produce an experimental setting combining people and technology in order to motivate innovation, 
development and research. Some examples of such environments and approaches are given in (Chin & 
Callaghan 2013), where "living labs”, “iCampus”, “smart box”, and Pervasive-interactive-programming are 
combined to produce a highly motivating and effective educational environment. 

Depending on the educational context, several approaches to teaching IoT can be adopted. At the lowest 
level are individual IoT devices, and understanding them requires the knowledge of electronics and low-level 
microcontroller programming. The middle level is informatics-oriented, encompassing communication 
protocols, system integration, web services, human interfaces, etc. At the highest level are the design and 
business aspects of developing IoT applications. Teaching IoT comes with a set of problems for both the 
students and the educators, especially when concentrating on the higher levels of IoT. The main problems are 
(Callaghan 2012): the lack of electronic design expertise among students; the need for complex hardware and 
software tools; the time-consuming nature that limits complexity; and student-built hardware usually has 
fixed functionality and too simple to give realistic product development experience. 

One approach in teaching IoT is the use of simulation tools to simulate the devices or the environment in 
which they are deployed. An example can be seen in (Yilmaz 2011) where the authors utilized a test card 
capable of processing digital and analog inputs and simulating home appliances, a model of a home, and a 
simple control interface with 3D models of house interior. In this approach, the teaching can be performed 
even without some or all of the hardware, and it allows the course to concentrate more on the software aspect 
of IoT. However, the simulation cannot replace the benefits of actually working with the IoT devices and 
potentially constrains the students’ imagination, making them think within the limits of the simulated 
scenario. 

Working in the wide field of IoT technologies can require skills such as problem solving, team work, and 
leadership, as well as practical experience with actual "things” used. The active learning approach is shown 
to be very effective in such conditions, and a mixture of collaboration, competition and peer learning in a 
hands-on environment reinforces the students’ transversal skills (teamwork, communication, critical analysis) 
(Panadero et al. 2010). 

The main aim of this paper is to improve the process of learning IoT using modern technologies and a 
hands-on approach. A model for conducting hands-on IoT classes with business informatics students, 
supported by a cloud infrastructure and web services is presented. The model relies on the existence of cheap, 
general-purpose programmable devices like Raspberry Pi microcomputer and the Arduino microcontroller 
boards. Using these devices, it is possible to easily produce slightly less efficient IoT solutions that do not 
require specialized components or expert knowledge to build. A pilot class was performed with a small group 
of interested students as a test of feasibility, and this paper, accordingly, represents a starting point for further 
research that will be performed on larger groups of students. 
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2. MODEL FOR TEACHING IOT 

The generic model of platform for teaching IoT is shown in figure 1 and describes the required equipment 
and supporting infrastructure. The model is split into three layers - the device layer, the service layer, and the 
app layer. Some components used in the actual implementation at the Department for E-Business of the 
Faculty of Organizational Sciences at the Belgrade University are also mentioned below. 

Ideally, the equipment for an IoT class should be as rich as possible, allowing the students to design a 
multitude of solutions. In reality, financial limitations can be severe, especially in poorer countries, and the 
equipment should be selected carefully. The main components at the device layer should be multifunctional, 
cheap microcomputers and microcontrollers, capable of taking many roles according to user programming. 
Such devices are readily available, the examples being the Raspberry Pi and Arduino which were used in the 
pilot implementation. Temperature, light, noise, and other sensors are usually simple and cheap, while 
expensive actuators and controllable appliances can be simulated by using diodes and simple circuits (Yilmaz 
2011 ). 

The operating logic can be distributed between devices, but an easier solution is to centralize it behind a 
well defined web service API. Devices that perform measurements (i.e. sensor nodes) can occasionally report 
to web services which take on the role of the middleware in the system. The devices can receive their 
instructions either by polling predetermined services, or by running their own web services for input if they 
are powerful enough. The web service approach is especially beneficial in an educational, cooperative 
environment. Students can develop their own services and share them with other students; more complex 
services can be built by integrating students’ services with cloud -provided and external web APIs. Client 
applications, whether web, desktop, or mobile -based, interact only with the web services and need not 
concern themselves with device-specific knowledge. 

The core component for IoT class implementation was the web hosting service freely provided to the 
students, which was implemented using the ISPConfig software allowing the deployment of student web 
services and web applications. Reporting of sensor readings was recognized as a common scenario in IoT 
applications, so a simple API was provided to the students for this purpose, allowing the storing, use, and 
sharing of sensor readings, as well as importing or generating external data for use in simulations of some 
specific conditions. SMS sending/receiving service was also provided to the students, and other services 
could be provided depending on the existing infrastructure of the educational institution. All services that 
need user accounts for their operation can, for improved interoperability, rely on the centralized store of 
student credentials. In the actual implementation, the OpenLDAP directory used by the learning management 
system of the Department was used for this purpose. 


*WS - student-developed web services 
*APP - student-developed web applications 
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Figure 1 . Model of infrastructure layers 
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The implementation of the presented model can potentially be very complex, with a large number of 
interconnected services, some of them possibly integrated with existing learning systems in order to allow 
easier tracking and grading of student activity. Although not necessary, an optimal solution in such 
conditions is to utilize a cloud platform, providing high scalability and redundancy, with better utilization of 
available processing resources (Beaty 2013; Sultan 2010). The suitability of utilizing cloud infrastructure in 
higher education is presented in a number of research papers (Despotovic-Zrakic et al. 2013; Ercan 2010). 
The pilot class relied on the existing, OpenStack based cloud infrastructure used by the E-Business 
Department. 


3. COURSE STRUCTURE 

The goal of the IoT pilot course is to introduce and educate students with a background in business 
informatics in using the hardware, operating systems, software, and tools for automation of smart 
environments. The course consists out of four units (figure 2): 

• Introduction to technologies used, 

• Defining scenarios, 

• Developing web services, 

• Developing web and mobile applications 

The students are first introduced to all of the elements of systems used for smart environment automation. 
This introductory part includes a review of hardware components, scenarios of their use, business models for 
their application, and successful existing systems. 

The course is realized mainly through laboratory exercises. The students should be divided into teams, and 
the exercises done in the form of workshops. During the semester, students should receive a specific task for 
every exercise. These tasks describe the context of application and the method for exercise completion. Every 
team should get a number of user requests for development of an intelligent environment. Based on these 
requests, the students need to envision a scenario and design two schematic representations showing hardware 
components that would satisfy these requirements. The first scheme should give a technical presentation of 
sensors and actuators and their connections to microcomputers and microcontrollers. The students can, for 
instance, use a free, open-source application called Fritzing for this purpose. The other scheme is to present the 
entire intelligent environment with the locations and interactions of larger units comprising the solution, and 
the students can create it using any standard software for drawing diagrams. 

The implementation of the hardware environment should be based on the previously designed scheme for 
interconnecting sensors and other devices. For this task, the students are to be provided with devices like 
Arduino microcontrollers. Raspberry Pi microcomputers, and various sensors, actuators, and other 
components. In order to make it easier for the students to review their knowledge at home, without actual 
equipment, a set of video clips showing different aspects of handling the hardware and other tools can be 
provided to them. 

After the implementation of the hardware infrastructure, the students should design and implement a 
software system to complement their previous solution. Arduino microcontroller uses a modified version of 
C++ programming language, while the Raspberry Pi platform best supports the Python programming 
language, and the students should utilize any previous knowledge available and work on those parts they are 
most familiar with. The students can utilize Xively in their projects, the public cloud service for collecting and 
accessing data from various sensors, or develop similar solutions smaller in scope. 

As previously mentioned, the exercises are performed in the form of workshops, where every team 
receives a specific context and a task to implement a scenario for use of smart devices in said context. The 
examples of some of the main contexts/tasks include the following: 

• Smart home context - Arduino or Raspberry Pi are to be used in combination with temperature sensor. 
If the temperature exceeds a certain value, an SMS message should automatically be sent to the owner 
of the smart home. Using a web or mobile application, the owner has a continuous insight into the 
measured temperature. 

• Smart classroom context - Arduino or Raspberry Pi are to be used in combination with an NFC tag 
reader and a digital display. During the entry into the classroom, every student should tap his NFC 
card on the reader, which will register his presence in the current lecture. Since the capacity of the 
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classroom is limited, if all seats are taken, the display outside of the classroom should show an 
appropriate notice. The students can use a web or a mobile application to check the number of free 
seats in the classroom. 

• Smart library context - for smart library automation, the students should utilize noise sensors and a 
speaker in combination with the standard Arduino or Raspberry Pi devices. If the noise sensors record 
an amount of noise above a certain limit, a voice message should be played using the speakers in the 
library. The administrators should be able to check noise levels using a web or mobile application. 

The final outcome of this course should be the integration of all laboratory exercises into a single project. 
The final grade is composed out of grades for tasks completed in exercises, the grade for an electronic test, and 
a grade for completion of the project, with the tasks and the project making up to 80% of the grade. The 
project should define user requirements, scenarios, project documentation, web services, and a web or mobile 
application for automation of the smart environment. 


Q 


Introduction to technologies used (14 classes) 

Introduction to Internet of Things 

Designing and implementing hardware systems 

Programming microcomputers and microcontrollers 




Developing web services (6 classes) 

• Designing and implementing web services for 

automatization of smart environments based on the 
previously defined scenarios 


0 


Developing web and mobile applications (8 classes) 

Designing and implementing web and mobile 
applications for automatization of smart environments 
based on the previously developed web services 


Figure 2. IoT course structure 


4. PILOT CLASS IMPLEMENTATION AND RESULTS 

The pilot course was implemented at the Laboratory for E-Business according to the course structure outlined 
in the previous chapter, with 8 students of master studies taking it as an elective class. All of the students 
were between 20 and 25 years of age, all had similar backgrounds in business informatics, with only slight 
variations in their area of specialization and interest, and that half of the students were employed in some 
form (part-time or full). The pilot course was shorter in length compared to the previously described structure 
and consisted out of 12 classes in total, four covering the introduction and remaining eight the other, practical 
sections. 

After the completion of the entire course, the students were presented with a questionnaire with questions 
grouped into five sections: demographic data, opinions about course structure and execution, opinions about 
workshop content, student’s motivation, and student’s perception of own knowledge. Students were also 
graded according to their completed tasks, project, and test, and out of eight students participating, three had 
the final grade 8, four had the grade 9, and one had the maximum grade of 10, for an average of 8.75. 
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The questions from the first group concerning the course structure and execution were to ascertain 
whether the course structure was well balanced, if the tasks were too difficult and the allotted time sufficient, 
and what was the students’ opinion of the interaction model between the students and teachers, as well as the 
interaction between students themselves. Here, 62.5% students thought that the optimal length of a single 
class within the course was three hours, while the remaining students approved the actual length used (1.5 
hours). All students, however, agreed they had sufficient time to complete the tasks they were assigned. The 
tasks were mostly marked as being of medium difficulty (87.5%), but most of the students (75%) thought that 
the assistant’s help was needed, and 25% thought it was necessary. All of the students agreed that the 
competitiveness between the groups was a motivating factor. 

The third part of the questionnaire was concerned with the actual content of workshops and was to 
ascertain as to which parts of the workshop were best liked by the students, which were considered the most 
difficult, and which were seen as the most important. These questions were to provide insight into the 
balance of the course. Accordingly, the activity of thinking up IoT scenarios was mostly ranked as average 
(62%), with all other answers going above that. The students also thought this activity was of average 
interestingness (50%), and that the proper understanding of the scenarios was very important for completing 
the given tasks. Most of the students didn’t have any preknowledge about IoT, but didn’t think that dealing 
with the hardware was too difficult. This part of the tasks was rated mainly as very interesting (50%), and a 
similar opinion was received concerning the importance of these technologies in the wider task. The students 
had, on average, best preknowledge in the domain of web services, since this was a part of their earlier 
education (50% marking it as average), but most also thought that dealing with services was of medium and 
high difficulty. This can be explained by the fact that it was expected for the students to have a certain 
preknowledge in this area, and the requirements were accordingly stricter. The students also thought that web 
services were a crucial component of their solutions (25% - important, 75% - very important). The main 
sentiment was that the Android development was not too difficult, and that this activity was very interesting 
(50%). Some of these results are given in Table 1. 


Table 1 . Questions concerning the perceived interestingness and difficulty of dealing with IoT hardware and web services 


Question 

Answer 

# responses 

Question 

Answer 

# responses 


Very boring 

0 


Very easy 

2 

How interesting was dealing 

Boring 

1 

How difficult was dealing 

Easy 

2 

with software and hardware 

Average 

2 

with software and hardware 

Average 

2 

components? 

Interesting 

4 

components? 

Difficult 

2 


Very interesting 

1 


Very Difficult 

0 


Very boring 

1 


Very easy 

1 

How interesting was dealing 
with web services? 

Boring 

1 

How difficult was dealing 
with web services? 

Easy 

0 

Average 

1 

Average 

2 

Interesting 

2 

Difficult 

3 


Very interesting 

3 


Very Difficult 

2 


The fourth part of the questionnaire was short, and contained questions about how the students were 
motivated to continue improving their knowledge of the technologies covered during the class. 75% of 
students were motivated to do so, with 12.5% motivated partially, and remaining 12.5% not motivated. 

The final part was concerned with the students preknowledge and about how they perceived their 
improvement in certain areas. Students did not have much knowledge about IoT beforehand, but had an 
average level of knowledge with programming and working with web services. The students mainly had only 
an elementary level of knowledge about Android OS, with only some of them having completed an elective 
course in Android programming. The students thought that the provided educational materials were sufficient 
(75%), and that they have gained a satisfactory amount of knowledge about IoT and other applied 
technologies. 

Additionally, the students were asked to give their opinions about the class and about possible 
improvements. All of the opinions were very positive, and some of them are given below: 

• “Intriguing, interactive, and fun. Motivating.” 

• “It’s very interesting to see the practical results of programming” 

• “It allowed me to review important things” 

• “There should be more classes like this. Working on a concrete example, whose state can be modified 
through PHP or Android was a top-class experience” 
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• “No objections. The workshop was interesting and motivating. Also, to see the results improves the 
dry programming experience a lot. 


5. CONCLUSION 

The presented model has several advantages - it is cheap, effective, scalable, and well-suited for students not 
coming from a hardware background. The model requires some supporting infrastructure and establishing a 
number of software services, but all well within the capabilities of any business informatics or software 
oriented curriculum. The students taking part in the pilot class were well-motivated, but the short class length 
and small number of participants did not allow us to make any general conclusions, and further research will 
be performed in future classes. 

The main deficiency of the pilot class was the fact that only eight students participated. Still, there is 
currently not many research papers concerning the course structure and execution of loT classes, and some 
general remarks, possibly useful for future research and larger IoT classes can be given based on the obtained 
results and experience. The students rated the help from the assistants as very important for the completion of 
their tasks; this especially needs to be considered if future classes are performed with a larger number of 
students. IoT exercises that attempt to encompass both the hardware and software aspects of IoT should 
likely be of longer length, as the students were mostly in favor of 3 hour classes over those twice as short. 
The SMS service that was provided to the students for sending and receiving SMS messages was very well 
liked, although it was not a core part of the IoT classes. This is probably the case since the SMS messages are 
a very familiar concept, and it allowed the students to envision IoT integration in a more realistic manner. 
The educational institution should therefore attempt to provide the students with as many other services as 
possible in order to widen the possibilities and increase student motivation. Students also felt that seeing the 
results of programming immediately was motivating, and IoT exercises should be designed in such a way to 
provide directly verifiable results, and preferably a direct impact on the physical world in some way. 
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ABSTRACT 

This paper considers how students deal with malfunctions that occur during the use of web conferencing systems in 
learning arrangements. In a survey among participants in online courses that make use of a web-conferencing system 
(N = 129), the relationship between a preference for internal or external locus of control and the perception of technical 
difficulties is examined. This study starts from the idea that the experience of malfunctions has an influence on the 
acceptance of information systems in education, while support services are availed mainly by users who act responsible 
in using technology anyway. 


KEYWORDS 

Malfunction, Technology Acceptance, Web-Conferencing, Webinar, Locus of Control 


1. INTRODUCTION 

People who are involved in teaching and learning often consider the use of an information system in terms of 
substitution. That is: A digital appliance replaces analogue practice. Especially online communication and 
collaboration - online courses in the realm of education - are compared with face-to-face meetings. Starting 
from the notion of “cyberspace” for the Internet, we are used to metaphors that compare applications of 
information and communication technology to known and familiar structures of the real environment 
(“space”) that surrounds us. In this paper we consider “virtual classrooms”, referring to the use of web- 
conferencing systems for synchronous online collaboration in education. 

In this introduction we are going to review the relevance of malfunctions that occur during the use of web 
conference systems for the acceptance of information systems in education. We then draw attention to the 
concept of locus of control, which might explain the prevalence of technical difficulties. In the second part of 
this paper, we present a study that examines the preference for internal or external locus of control 
concerning technical difficulties in online courses. A conclusion ends this paper, which presents some 
insights for support services in online education and questions for further research. 

1.1 Technology Acceptance and Malfunctions 

It stands to reason that a comparison between two forms of collaboration is followed by a preference for one 
or the other option. In a comparison between two ways of interaction and conversation, people are likely to 
consider the additional benefits of each alternative. They may also deliberate on efforts that can be avoided or 
even may imagine opportunities that arise from a combination of both alternatives. The individual weighting 
of single factors, the evaluation of specific criteria and even expectations concerning the options at stake are 
rarely purely rational. Nevertheless, the acceptance of a new, technology-based form of communication and 
collaboration can be explained by a cost-benefit analysis. Some values justify the use of technology for 
teaching and learning, for example, the avoidance of expenses for travel, or an increased benefit such as 
better learning opportunities - or, maybe, just the fun to make use of a new technology. 
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Furthermore, it may be assumed that the need to ensure the correct operation of the technical equipment 
adds as an expense in this cost-benefit analysis. The provision for a flawless operation requires time, and 
material as well as personnel resources. Personnel resources include knowledge, experience and problem- 
solving strategies. Malfunctions, breakdowns or system failures interrupt or diminish the benefits of 
technology use. In addition, malfunctions cost time for troubleshooting and fault elimination. Malfunctions in 
web conferencing are particularly burdensome. As opposed the use of Internet technologies in asynchronous 
mode (for example, in online courses with common learning management systems, but also for online 
education with services and applications of Web 2.0), synchronous online collaboration (employing video 
and audio conferencing as well as a shared workspace or a shared desktop) is likely to stop when one 
participant encounters a technical problem. Technical malfunctions in web-conferencing often result in poor 
audio quality, e.g. noise or echo in voice transmission, unbalanced sound volume, or even in acoustic 
feedback (Larsen effect). Frequently, video pictures from some participants are missing, due to a wrong set 
up of their webcams or low bandwidth. Low bandwidth or server overload may also result in high latency, 
which confounds the notion of real-time collaboration in a joined workspace. 

In addition, web-conferencing systems are quite complex. The average user is unable to evaluate the 
reason for malfunctions. In web-conferencing, insufficient sound quality may be due to server load, 
bandwidth of the Internet connection of single participants, the encryption in the domestic Wifi network, or 
even hardware of one participant, like cable, microphone and speakers. It is noteworthy that most of the web 
conferencing systems from the current generation provide a functionality to integrate a conference call on 
landline telephones in the meeting, in order to ensure hassle-free conversation. The landline telephone 
network is considered to be both easy to use and extremely reliable. 

And although landline telephony nowadays is based on a global technical system that is, like the Internet, 
digital and packet-oriented (cf. Janevski, 2014, S. 23ff), we take landline telephones “for granted”, that is: 
They are considered to be part of the “real world”, and not part of “cyberspace”, just like most of artefacts 
that surround us in everyday life as well as in learning spaces. As long as walls, tables, pens or projectors 
fulfil their function without difficulty, they disappear from consciousness. Only a malfunction, or a 
“breakdown” (with reference to Heidegger cf. Winograd & Flores, 1986, S. 68ff), discloses technology. 

Neither research in instructional design nor development in technology enhanced learning has addressed 
malfunctions in technical systems, their impact on teaching and learning or their relevance for the acceptance 
of digital technology in education broadly. More generally, there is only few research on anger and 
frustration in dealing with software and computers (cf. Bessiere, Newhagen, Robinson, & Shneiderman, 
2006; Charlton, 2009). Ergonomic requirements for the use of software applications name “error tolerance” 
and “conformity with user expectations” as two out of ten design principles (ISO 9241-110, see Schneider, 
2008). Usability evaluation is oriented at these principles, as well as their relevance to perceived usability is 
investigated (cf. Pataki, 2009, S. lOOff). However, current models on technology acceptance (TAM3 as a 
prominent example, cf. Venkatesh & Bala, 2008) do not explicitly incorporate negative, hence frustrating 
user experiences as explanatory factors. Malfunctions are likely to affect both “perceived usefulness” and 
“perceived ease of use”; but still, in TAM3 these factors are explained by external determinants (like 
subjective norm) and internal determinants (like computer-related self-efficiency), not by frustrating events. 

Obviously, research and development in technology enhanced learning is based on an affirmative attitude: 
Since everybody wants to demonstrate how things work, reports on failure are hard to publish. Therefore, 
from a technical perspective malfunctions are generally regarded as a technical problem, which in turn is to 
be solved with more or better technology (cf. Morozov, 2013). Education however, should take into account 
how students adopt technology. The efforts to care for the hassle-free operation are only one part of the story. 
Becoming aware of personal responsibility in dealing with technology is a prerequisite for the responsible 
use of technology, and as such, an important part of media literacy. 

1.2 Locus of Control and Malfunction 

There is an important differentiation for efforts in dealing with malfunctions: time, knowledge and problem- 
solving strategies for coping with malfunctions are individual resources; while technology itself, 
infrastructure as well as staff for maintenance and support are usually beyond control and influence of the 
individual user. The concept of locus of control suggests itself here (cf. Rotter, 1954, S. 105ff). In differential 
psychology, we may differentiate between an internal locus of control and an external locus of control, that 
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is: People are more or less convinced that events are the consequence of individual action (not necessarily of 
their own), or that events can not be influenced by individual action. In various fields of research, this 
distinction was further elaborated towards a multidimensional model. Interestingly, relevant research is rather 
concerned with failure than with success. The notion of locus of control comes into play when the avoidance 
of damage or the improvement of pain is addressed. There is considerable research in the field of health, 
illness and recovery, including the Multidimensional Health Locus of Control Scale (MHLC) regarding 
individual attitudes and beliefs towards illness and healing (cf. Wallston, Wallston, Kaplan, & Maides, 1976). 

For a technology-related field, some studies on risky driving behaviour are apposite to the question of this 
paper. For issues related to road accidents, there is a scale measuring dimensions in locus of control 
concerning traffic (cf. Ozkan & Lajunen, 2005). The starting point here is the assumption that an internal 
locus of control in terms of traffic accidents is a key prerequisite for drivers to avoid risky driving. The 
named study differentiates the external locus of control between technology (e.g., the condition of the 
vehicle, the condition of the road) and fate; as well as the internal locus of control beliefs are distinguished 
between self and other road users. This structure allows an analogy to the study presented in this paper. The 
use of web conferencing systems is a complex technical system that requires a well operating infrastructure 
(servers, broadband Internet), while disturbances are hard to assess and therefore may appear to be fateful. In 
addition, multiple users contribute to the success of communication and collaboration; hence users might 
attribute responsibility for error-free operation to themselves or to their peers. 

1.3 Research Questions and Hypotheses 

The field of the present study is web-based synchronous collaboration. We examine web-conferencing in the 
context of online education, using Adobe Connect™ for course sessions, also known as “virtual classroom” 
or “webinars”. The objects of the investigation are the dimensions of locus of control, and their relation to 
technical malfunctions. In view of the above, the following questions arise: 

■ Is it possible to assess different dimensions of locus of control with respect to malfunctions in 
collaborative, web-based work environments? 

■ Is there a connection between the locus of control and perception of technical difficulties? 

Accordingly, the working hypotheses are that locus of control beliefs for coping with technical 

difficulties have a dimensional structure, and that, in view of differential psychology, there is a relationship 
between the dimensions of locus of control and the perception of technical difficulties. 

An educational relevance of this relationship arises from the very pragmatic question of suitable support 
services as well as from issues related to the acceptance of web-based synchronous collaboration in online 
education. As already mentioned, the questions that have been raised are also relevant with respect to media 
literacy, since internal location of control beliefs are a prerequisite for responsible handling of technology. 


2. INVESTIGATION 

As part of a survey on the acceptance of web-conferencing systems in distance education (cf. lunge, Klebl, & 
Mengel, 2011) students of the University of Hagen (N = 129) have been asked about the Level of Perceived 
Technical Difficulties while attending synchronous online sessions (i.e. “webinars” with audio and video 
communication as well as with shared presentation or shared workspace). In the following, we present the 
research design and the research instruments, and then outline the sample. In a third subsection, the results of 
the statistical analysis are presented. The paper concludes with a summary of results and an outlook to 
follow-up research in the last section. 

2.1 Research Design and Instruments 

The survey was conducted via an online questionnaire (in German language). Only students that had already 
participated in one or more web-conferencing sessions using Adobe Connect™ were invited. As part of a 
planned analysis to the Extended Technology Acceptance Model (TAM3, cf. Venkatesh & Bala, 2008), the 
questionnaire comprised scales for Computer-Related Self-Efficacy , Computer Anxiety , and Computer 
Playfulness. Based on existing survey instruments, new items had been developed for the presumed 
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dimensions of Locus of Control ( Self Other , Technology, Fate, Authority), which are essential for the study 
presented here. 

Likert scales were used for all the latent variables mentioned so far, with three or four items per each 
factor based on a reflective measurement model. In order to provide the opportunity to express an indecisive 
attitude, five response options were offered (from “totally agree” to “totally disagree”). The coding was 
carried out by numbers 1 to 5, where a low value represents a high level of agreement with the item 
statement. The online questionnaire represented the response options horizontally, with equal distance 
between the radio buttons. As generally accepted for Likert scales, we consider the response options as an 
equidistant scale and treat the resulting manifest variables as interval-scaled. 

In addition to these factors, the Level of Experience with the web-conferencing system Adobe Connect™ 
is of importance for the further report of the study. Therefore, we asked for the number of sessions where the 
respondents had participated. In the following subsections, we give more details about the latent factors. 

2.1.1 Level of Perceived Technical Difficulties 

Some item statements concerning the Level of Perceived Technical Difficulties addressed the time factor (e.g. 
“Due to technical problems, sessions with Adobe Connect are unnecessarily prolonged.”) In addition, items 
considered cognitive load in relation to technical difficulties (e.g. “Occasionally, the distraction caused by 
technical errors outweigh the benefits of Adobe Connect.”). The four items used result in a consistent scale 
(Cronbach's a = .88). On average, the level of technical difficulties is perceived as neutral (x = 2.8) and has a 
reasonable variance (SD = 0.9). 

2.1.2 Locus of Control: Self, Other, Technology, Fate, Authority 

Item statements for Self Fate and Authority as dimensions of Locus of Control were adapted from the 
German version of the Multidimensional Health Locus of Control Scales (MHLC), according to Muthny & 
Tausch (cf. 1994). However, four items were selected from six. They describe aspects such as responsibility 
(e.g. for Self, “I am responsible for ensuring that the session using Adobe Connect works well.”) or 
compliance (e.g. for Authority, “The best way to work smoothly with Adobe Connect is to follow the 
instructions exactly.”). In order to measure locus of control beliefs in terms of Other, various aspects such as 
responsibility, carefulness or behaviour were transferred from item statements related to Self (e.g. “Only if all 
participants have the required technical skills, a meeting with Adobe Connect can operate smoothly.”). 

Statements about Technology were only loosely inspired by the English questionnaire for Traffic Locus of 
Control (T-LOC, cf. Ozkan & Lajunen, 2005). The transfer considered abstract concepts of complex 
technical systems (such as infrastructure, in T-LOC “dangerous roads”, for web -conferencing “Often, there is 
something wrong with Adobe Connect since the Internet connection is interrupted.”). The locus of control 
beliefs are assessed object-related, that is: Each item statement comprises or implies a reference to the 
employed web-conferencing system Adobe Connect™. 

A confirmatory factor analysis was executed in order to validate the presumed dimensional structure 
(extraction of factors by Principal Component Analysis, Varimax rotation with Kaiser Normalization). 
However, not all dimensions emerged in the factor structure clearly while involving all twenty items. In 
particular, the items related to the influence of the Authority do not form a consistent dimension and mingle 
with the items for Self. A second, exploratory factor analysis that excludes seven items without clear relation 
to a latent factor is apt to validate the dimensions Self, Other, Technology, and Fate with the remaining 
thirteen items. Conceptual considerations and a final audit of internal reliability result in a dimensional 
structure with four items for these four dimensions of locus of control (with .55 < Cronbach's a < .85). The 
dimension of the Authority was not longer considered in the following evaluation. The locus of control 
beliefs for Self, Other, and Technology are rated on average slightly more affirmative (2.7 < x < 2.9), for Fate 
slightly more negative (x = 3.5). The factor Technology has a lesser variance (SD = 0.6) than the other 
dimensions (0.7 < SD < 0.8). 

2.1.3 Computer-Related Self-Efficacy, Computer Anxiety, Computer Playfulness (Anchor 
Factors) 

The item statements of these latent factors match an adequate translation of the anchor factors from the 
Extended Technology Acceptance Model (TAM3, cf. Venkatesh & Bala, 2008). There, they form a general 
framework for the acceptance of specific software applications, supplemented by Perceived External Control. 
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In contrast to the latent variables for the locus of control scales these anchor factors are operationalised 
without reference to any specific information system. For all three latent factors, the respective four manifest 
variables generate a consistent scale (with .71 < Cronbach's a < .86). The respondents rated Computer 
Anxiety on average rather low (x = 4.4), while in contrast they assessed their Computer-Related Self-Efficacy 
and their Computer Playfulness rather high (x = 1.9 and x = 2.2). 

2.1.4 Level of Experience 

Three groups for Level of Experience were formed based on the requested number of meetings in which the 
respondents participated. A separation between once , several times (2 to 3) and frequently (4 or more) 
resulted in approximately equal groups (N = 45 , 41, 43). 

2.2 Sampling Design and Data Collection 

Data were collected via an online questionnaire. For pragmatic reasons, students that participated in online 
courses using the web-conferencing system Adobe Connect™ were asked to participate in the study. To this 
end, lecturers were asked to send an e-mail with an invitation to the students who have participated in their 
course between or shortly after the lessons. Therefore, it is not known how much students actually were 
asked to participate; hence it is impossible to give a statement about the return rate. In addition, conclusions 
about effects of cluster sampling are not possible, because it can be assumed that individual students had 
participated in several courses and, therefore, were invited twice or more to participate in the study. 
Participation was rewarded by inclusion in a prize draw. Participation in the draw was voluntary and 
anonymous by means of an identification number. 

2.3 Results 

A total of 133 students participated in the survey and fully completed the questionnaire. Due to implausible 
data (in particular on the number of meetings in which the respondents have participated) four records were 
excluded. Of the remaining 129 respondents, three-quarters were female (76%), one quarter male (24%). On 
average, participants were 36.8 years old. The standard deviation in age was 8.1 years, with an age range 
from 21 to 59 years. Most students are enrolled at the Faculty of Cultural Studies and Social Sciences (76%). 
The remaining participants study at the Faculty of Mathematics and Computer Science and at the Faculty of 
Law. 

2.3.1 Relationship between Locus of Control and the Level of Perceived Technical Difficulties 

Starting from the working hypothesis, that there are different dimensions of locus of control in coping with 
malfunctions, and that these different locus of control beliefs influence the perception of technical 
difficulties, the relationship between the dimensions of locus of control beliefs and the level of perceived 
technical difficulties has been investigated. Of course it is impossible to draw a conclusion of cause and 
effect from a statistical correlation at the first attempt. 

A Pearson’s product-moment correlation for each dimension in Locus of Control ( Self Other , 
Technology, Fate ) results in moderately strong correlation coefficients that are each highly significant 
(p < .01). There are the negative correlations for the internal dimensions ( Self r = -.38; Other r = -.29). 
Students who experience fewer technical difficulties are more likely to believe that they or others can 
influence successful operation. As expected for the external dimensions, the correlations are positive (for 
Technology r= .41, for Fate r= .55). People who tend to assume that errors in technology or just fortune 
determine successful operation experience more technical difficulties. 

In addition, the interactions have been tested by a regression analysis, using the Level of Perceived 
Technical Difficulties as the dependent variable. A linear regression analysis which includes all four 
dimensions as independent variable in the model results in an explained variance of 33%. In this case, only 
the regression coefficient for the external locus of control beliefs are significant (p < .05) and confirm the 
positive relationship (for Technology 6 = .34; for Fate fi = .47). 
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2.3.2 Relationship between Anchor Factors and the Level of Perceived Technical Difficulties 

It raises the question whether more general psychological dispositions in relation to the use of computers 
determine the perception of technical difficulties, in contrast to the specific beliefs of locus of control related 
to the particular web-conferencing tool in this study. With the anchor factors of TAM3 ( Computer-Related 
Self-Efficacy, Computer Anxiety, and Computer Playfulness), the questionnaire assessed attitudes towards 
computer in general, which may also determine the perception of technical difficulties. 

A Pearson’s product-moment correlation for each Anchor Factor in relation to the Level of Perceived 
Technical Difficulties demonstrates no significant correlations. Also, the result of a linear regression analysis 
with the Level of Perceived Technical Difficulties as the dependent variable yields no relation. Only about 
5% of the variance in perceived technical difficulties is explained by the anchor factors. For the present 
study, no influence of general attitudes toward computer usage on the perception of technical difficulties can 
be found. 

2.3.3 Relationship between Locus of Control and Anchor Factors 

Furthermore, the dimensions in Locus of Control are independent from the general attitudes toward computer 
usage as well. The matrix of Pearson’s product-moment correlation between the dimensions in Locus of 
Control (Self, Other, Technology, Fate) and the Anchor Factors of TAM3 ( Computer-Related Self-Efficacy, 
Computer Anxiety, and Computer Playfulness) reveals no significant correlations, with only one exception. 
There is a low, but significant correlation between Computer Playfulness and the Locus of Control for Other 
(r = .22 at p < .05). Some of those who like to experiment with the computer are slightly inclined to shift 
blame for any malfunction in computer-based interaction to others. Nevertheless, from the present study it 
can be assumed that the dimensions of locus of control are independent constructs. 

2.3.4 Level of Experience as a Possible Cause 

One question that remains unanswered so far is whether different control beliefs actually affect the 
perception of technical malfunctions, or, on the contrary, a repeated experience of technical malfunctions 
causes people to develop an attitude of powerlessness. Who often experienced that technology does not work 
will be inclined to value own capabilities inferior to technical operations and fate. Reversed: Who is 
convinced that the responsibility for error-free operation lies with the users, will take action to prevent 
malfunctions. 

The present study may provide a clue how to explain the relationship between Locus of Control and Level 
of Perceived Technical Difficulties. It is assumed that it makes a substantial difference, whether a user uses a 
tool for web-conferencing once (for a typical session of a maximum of 120 minutes), several times (several 
sessions over at least a week or two) ox frequently. The more often the tool is used, the more likely are 
malfunctions. At the same time users get used to the system and gain some skills and knowledge in 
troubleshooting, especially when users tend to internal locus of control beliefs (Self, Other). 

A review of the differences between the groups that had taken part in sessions with the web-conferencing 
tool once, several times ox frequently does not result in significant mean differences, neither for the Level of 
Perceived Technical Difficulties, nor for the dimensions in Locus of Control (One-way ANOVA, equal 
variances assumed, post-hoc test with Tukey-HSD). 

It may therefore be concluded for the present study that both the dimensions in Locus of Control in 
coping with malfunctions as well as the Level of Perceived Technical Difficulties remain stable over a certain 
period of time. In this short term view, experience does not affect locus of control beliefs. Under the 
assumption that frequent use cause more technical errors, it may be concluded that it is not the perceived 
technical difficulties which affect the locus of control beliefs, but vice versa. 

2.3.5 Gender Differences 

Despite the unbalanced distribution of men and women in the sample, an analysis on differences between the 
sexes is advised. In general, there are no significant mean differences for the dimensions in Locus of Control. 
However, women tend to perceive more technical difficulties (Level of Perceived Technical Difficulties: x 
= 3.3 for women compared with men x = 2.7, t- test p < .05). Therefore, differences between men and 
women contribute to the variance in the perception of technical difficulties. It is not clear what disposition 
may provide an explanation here, since there are no significant mean differences between the sexes for the 
Anchor Factors of TAM3 (Computer-Related Self-Efficacy, Computer Anxiety, and Computer Playfulness). 
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3. DISCUSSION 

The study presented in this paper confirms the assumption that it is possible to differentiate dimensions of 
locus of control beliefs for coping with malfunctions in web-based collaborative work environments. These 
locus of control beliefs are, on the one hand, internal , relating to human beings as responsible and capable: 
The participants in the study consider it more or less important, what they or others do to avoid malfunctions. 
On the other hand, locus of control beliefs are external, that is: The respondents consider it more or less a 
matter of technology or fate whether something works or not. 

It can also be shown that these locus of control beliefs are associated with the perception of technical 
malfunctions. Who has an internal locus of control faces less technical difficulties. Who is convinced that the 
influence of human beings is limited perceives more technical difficulties. By comparing with the level of 
experience in using the web-conferencing tool in question it can be concluded that negative experiences of 
technical difficulties do not affect the locus of control beliefs. Hence, locus of control beliefs guide the 
perception of malfunctions. These locus of control beliefs are more general than self-efficacy expectations. 
That is: There is no salient difference between the attribution of responsibility to oneself or to others; just as 
there is no important difference between the consideration of technology and destiny. The decisive difference 
lies in the attribution of responsibility to human actors as opposed to non-human factors. 

Starting point for the present paper is the assumption that malfunctions of technology in teaching and 
learning add to the individual cost-benefit analysis concerning the usage of information systems in education. 
On the basis of the above, it can be concluded that the perception of technical difficulties is significantly 
determined by different dimensions of locus of control. However, the study presented above has not 
investigated the direct relationship between malfunction and acceptance. This connection is quite plausible, 
but not proven as yet. 

Given the model of technology acceptance TAM3 (cf. Venkatesh & Bala, 2008), it stands to reason that 
experiences of malfunctions have influence on “perceived ease of use“ as well as on “perceived usefulness”, 
and thus have impact on “behavioural intention to use”. Here, further investigations should assess the 
relevance of failure and frustration. It would be interesting to examine the time course of frustration and 
competence in the context of locus of control beliefs. We suggest a U-shaped curve of competence 
experience in coping with malfunctions: Initial difficulties may lead after a few sessions to an experience of 
powerlessness. After a while, users acquire routines to manage these difficulties or to avoid them. This leads 
to a competent and self-responsible use. 

The present study is approached from the perspective of differential psychology, which underlies the 
concept of the dimensional structure of locus of control beliefs. Different people have different preferences, 
to which they attribute responsibility for an error-free operation. These preferences - internal or external 
attribution; to Self, Others, Technology, or Fate - is of importance for preparing and supporting the use of 
technological devices and applications in communication and collaboration. It can be assumed that support 
services will be accepted from those persons who have an internal locus of control, in particular for the 
dimension of Self. Who believes that the responsibility for a trouble-free session lies with herself or himself, 
will read the instructions needed to prepare everything well and is likely to develop routines for trouble- 
shooting, in order to experience less technical difficulties. 

However, those who are inclined to believe that nothing can be done anyway, because technology simply 
is supposed to work (and if not, this is due to defects or misfortune), will not be accessible for instructions 
and help during preparation. The result could be described as “preaching to the converted” - more often than 
not observed by support stuff: Any services reach only those who are already in a position to act responsibly 
with technology. 

The question remains about how to help and support those who currently take little or no responsibility 
for the prevention of malfunction. From the concept of locus of control, it can be concluded that services that 
fully exonerate the user are indeed well received by people with external locus of control beliefs, but then 
require more and more support, since they do not encourage self-reliance. Therefore, support service should 
promote personal responsibility for technology use. This can be done, for example, by setting goals for an 
online course, targeting an undisturbed web-conferencing session by evaluating malfunctions and praising 
circumspection, know-how and responsibility. The factors that foster self-reliance in adequate settings could 
be the subject of further research and development. 
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ABSTRACT 

In this society of digital environment, to keep pace with the technological change that causes the difficulties of 
information access together with royalty fees remunerated for making use of copyrighted materials. Creative Commons 
(CC) license is introduced. However, debates on the issues of copyright and CC license have widely arisen. This research 
is aimed to prove as to whether CC license destroys the balancing interests between the right holders and the public at 
large, including educators, through the fair use doctrine. Specifically, it is conducted to explore the legal strategy of CC 
license and analyze its potential for enhancing the sharing, distribution and reuse of creative works, especially in the 
digital era. The research findings suggest that the CC license can alleviate some of the problems caused by the copyright 
conflict and this can be helpful to the public, with no exceptions to educators. Moreover, some interesting 
recommendations are proposed to uphold the great benefits of CC license to be adopted in line with the copyright law. 


KEYWORDS 

Copyright, Creative Commons License, Digital Age, Educators. Information Access 


1. INTRODUCTION 

Due to the rapid change of technology, there is a huge demand for free images and music for educational use in 
schools, with many people just copying anything from the Internet without asking for permission or acknowledging 
the source. The right holders thus intentionally create a technological measure to restrict copyright material access. As 
a result, knowledge transfer through information access is unreasonably prohibited. Many people are of their views 
that as digital technology thrusts complexity upon copyright law, conflict has escalated between copyright holders 
desperate to institute a vigorous enforcement mechanism against copying in order to protect their ownership and 
others who underscore the importance of public interests, including those in the educational sector, in accessing and 
using copyrighted works. 

As the copyright movement grows, however, other movements are also gaining momentum - and fast. The 
copyright regime in many countries is considered to be too rigid, onerous and expensive to facilitate the exchange of 
ideas and encourage creativity. Instead, it is seen as protecting the monetary interests of those who create works 
without incentivizing the sharing of those works with the community so that they can be improved upon and added to. 

Legal scholars, users of protected works, and some creators have criticized the current copyright regime for its 
overreaching scope and duration. Different technical protection measures that restrict certain uses of works, legally 
backed by international copyright treaties and domestic copyright law, have increased concerns that copyright is 
developing into becoming a general regulation on the use of information. This development and copyright’s starting 
point of giving automatic exclusive rights to cultural objects have spurred counter movements into existence. 
Technology has been an important driving force of peer production development. It promotes practices and tools that 
encourage sharing, openness and peer production. Meanwhile, social structures have also required revisions. The 
default copyright system that is based on automatic exclusivity does not serve collaboration as such. Even though the 
communities have not lobbied for new legislation, they have created community norms that are implemented into 
copyright licenses and community guidelines. Collaboration is therefore hindered. Creative Commons (CC), set up in 
2001 by Lawrence Lessig, is an organization that works in tandem with existing copyright regime, but allows those 
who wish to license copyright in their works while reserving only the most basic of all rights, to do so within the legal 
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framework. It does this by providing what are called “Creative Commons license” (CC license) free-to-use and by 
providing information and tutorials on its use. 

Nonetheless, CC license has been the subject of heated debate, seeing great success and enthusiastic converts as 
well as staunch opponents and troublesome legal quandaries amongst both policy-makers and users. CC license is 
both praised and criticized for attempting to strike a balance between the protection of intellectual property and the 
need to encourage and foster collaboration; the philosophical foundations of the enterprise have been both lauded and 
lambasted, making the task of trying to understand what role CC license can play in today’s world an exceptionally 
interesting one. 

Many users and scholars are of their views that CC license is an extension of Copyright. CC license gives the 
creators the ability to dictate how others - users may exercise the creators’ copyright rights, allowing such users to 
copy the copyrighted works, make derivative works, distribute them, and so on. In terms of educational development, 
particularly in the digital era, CC license extremely opens new opportunities and ideas to both teachers and students. It 
also really helps to expand on copyright and encourage stakeholders to use materials in ways the creators are 
comfortable with. More importantly, there is arguable that the introduction of CC license may diminish the value of 
fair use doctrine under the copyright system. Thus, the researcher would like to prove as to whether CC license 
destroys the balancing interests between the right holders and the public at large, including educators, through the fair 
use doctrine. Specifically, this research is conducted to explore the legal strategy of CC license and analyze its 
potential for enhancing the sharing, distribution and reuse of creative works, especially in the digital era. 


2. THE NATURE OF COPYRIGHT AND CC LICENSE 

As the matter of fact, copyright law and CC license play different roles but CC license does not contravene the 
original purpose of copyright law as well as fair use doctrine. Copyright law protects any creative works since it is 
expressed into tangible and fixative form. Nobody can legally use such copyrighted works beyond fair use doctrine 
without a license. On the other hand, CC license serves as a license that people who want their works to be shared can 
issue. Certainly, such creative works are protected under copyright law - they are copyrighted works for being shared 
and everyone, encompassing educators, can make use of them under the conditions set up through licensing system. 

More descriptively, it is simply realized that the default setting of copyright system is all rights reserved - 
exclusive rights, which are generally so called. Users are required to get the permission to access the exclusive rights. 
License agreement is a significant tool for the right holders to screen the users to make use of their copyrighted 
materials. Users are required to acquire a license for every use of a work that is protected by these exclusive rights 
unless the exception of fair use is applied. The barriers to access are, thus, effectuated by two separate aspects of 
copyright law: first, the legal right to restrict access and to apply for injunction in case of unauthorized use, and 
second, information costs associated with securing a license (Elkin-Koren 2006). Certainly, royalty fees remunerated 
to the right holders often become an issue, particularly in the educational realm where non-profit purposes dominate. 
Accordingly, accessibility of copyrighted works can be continuously going down. Undoubtedly, copyright 
infringement at the same time is dramatically increasing due to circumvention of access costs under the territory of 
licensing agreement. CC’s strategy presupposes that minimizing information access costs is critical for enhancing 
access to creative works. It seeks to reduce these costs by offering a licensing platform. The strong copyright 
protection has fueled counter movement led by CC to introduce permissive and royalty free licenses. 

CC is a non-profit organization, which operates licenses to reduce the legal costs associated with the use of 
copyrighted works, especially for non-profit purposes to engage in creative enterprises, so free use of creative works 
can be promoted. This initiative operation is using licensing agreement for the purpose of strengthening the public 
domain. With respect to the goal of increasing access to and sharing of intellectual property, CC considerably allows 
copyright holders to release some of their exclusive rights while retaining the other. 

Respecting fair use doctrine and CC license, they are different. Fair use is described as a limitation and exception 
to the exclusive rights granted by copyright law to the authors or right holders of creative works (Fair Use 2014). It 
has been developed over the years as courts tried to balance the rights of right holders with society's interest in 
allowing reproducing in certain, limited, circumstances. Considering CC license, it has recently been launched to 
expanding the range of creative works available for others to create upon legally and to share. Thus, some scholars are 
of views that "CC is possibly the best publicized of the efforts to create an artificial public domain through contracts, 
to compensate for badly eroded zone of copyright-free work" (Aufderheide & Jaszi 2011, p.12). In the event where 
copyright law gives right holders the exclusive rights, fair use of copyrighted works is provided to balance the 
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interests of right holders and the public. However, whether or not the reproduction is within the boundaries of fair use 
depends on the fact of particular circumstances. It is simply said that fair use is a balancing test. This means that a set 
of requirements or circumstances - the four factors (U.S. Copyright Office 2012) - must be fulfilled for the exception 
to apply. Additionally, the four fair use factors work differently. Every fair use must fulfill some standard in regard to 
each factor. In brief, fair use allows limited reproduction of copyrighted work certain uses. On the contrary, under the 
territory of CC license, users are eligible to have extra rights to use the contents. Nevertheless, CC license does not 
replace or diminish fair use, but if users want to do more than fair use allows, they can examine the terms of CC 
license in order to consider what it permits and they are required to do in return subject to the terms used. In other 
words, CC license is not an attempt to supplant or to be better than fair use but it is, rather, an attempt launched as a 
special aid. 


3. COPYRIGHT AND CC LICENSE IN THE DIGITAL AGE 

The digital environment, which significantly reduced the cost of communicating and sharing works, has enabled new 
modes of production and distribution of information. Digital networks allow dissemination of works to a wide range 
of users at a very low cost, thereby reducing the role of some traditional intermediaries (such as the recording 
industry), while introducing new intermediaries (such as search engines) into the dissemination process (Elin-Koren 

2005) . Digital networks further enable the development of content, such as text and software, through collaborative 
efforts by individuals, involving educators, who interact and communicate with one another, often without any claim 
for exclusive rights in it. An example to illustrate this is an initiative in Argentina called “Encipel”, which is an 
educational community for creating documents that can be reviewed and improved by the community to share 
knowledge (OER Case Studies 2013). 

However, current copyright regime can be considered as a major obstacle for creative activities. First, people are 
concerned about the rapid expansion of copyright in recent years. The inefficiencies created by expansive copyright, 
by restricting access to information and restraining the ability to creation, are becoming an interesting issue that needs 
to take into a consideration. Secondly, the public are very much concerned with the shrinkage of the public domain 
while they are also enthusiastic about the open competition and free culture. They wish to safeguard the public 
domain and encourage sharing and reusing of creative works by individual creators. Lastly, it seems that many of the 
new opportunities that were made possible by the digital technology are increasingly reaped by the massive enclosure 
of the public domain and the increasing commodification of information (Elin-Koren 2005). 

The proprietary regime in recent years covers more copyrighted works in both analogue and digital forms. It 
affords protection to types of works, or new aspects of works, that used to be in the public domain, for instance, 
copyright and neighbouring rights afford protection for facts and mere data (Reichman & Uhlir 2003). The copyright 
bundle of rights has been expanded and covered a wider range of uses; for instance, the right to prevent unauthorized 
access to works in digital format which brought out the introduction of provisions concerning technological protection 
measures (TPMs). TPMs are a form of Digital Rights Management (DRM). DRM is the term given to various 
technologies used to control access to digital works or devices, to protect copyright in those works or the works used 
on the devices. For example, the iTime store incorporates DRM into its music, so that any music purchased from 
iTunes can be downloaded to a PC, burnt to a CD or downloaded to an iPod, but not downloaded and played on an 
MP3 player. 

In addition, copyright law creates relatively high information costs, due to the nature of copyright subject matters: 
non-tangible assets. In order to facilitate the right holders’ incentive in creating intellectual works to support the 
public’s knowledge attainment, the law provides remuneration in a form of royalty fees to compensate their 
endeavour. Every property right imposes information costs related to ascertaining the contours of legal relationship 
pertaining to the owned asset and determining the boundaries of goods to which it applies (Guibault & Hugenholtz 

2006) . In the case of copyright, these costs tend to be prohibitively high due to the fact that rights in creative works are 
not intuitive; the right holders are required to create their works so that they are granted the exclusive rights over such 
creative works. However, these high costs can be disadvantageous to educators whose purposes are not for profits. 

Copyright law made it difficult to take advantage of the new opportunities offered by digital networks. If licensing 
costs and legal exposure to copyright liability remain the same, creating and distributing online becomes expensive, 
notwithstanding the low cost of production and distribution. Furthermore, the proprietary regime has a tendency to 
further colonize other ways of producing content (Elin-Koren 2005). If one has to purchase a license to use someone 
else’s work, he/she is more likely to release his/her output under restrictive terms, either to comply with the license of 
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the underlying work, or to recover the cost of creation. This makes it difficult to create outside the proprietary model 
and thereby forces that model into the copyright one. This creates a chilling effect on individual creation. Legal 
counseling and licenses are more affordable and accessible to business. Thus the legal complexity created by the 
copyright regime gives businesses an advantage over individual creators, including educators. 

CC license allows the public to share their information on the Internet without being accused of copyright 
infringement. It advocates the use of copyright in a different way that would ultimately change its meaning. The 
strategy of CC license not only creates a public domain, but also encourages free of any exclusive proprietary rights 
under the intention of copyright regime. It responds to the problems in copyright law particularly in the digital 
environment, concerning the ambiguity between licensing agreement and contract as well as provisions regarding 
Technological Protection Measures (TPMs) that are too much restrictive to encourage the public to access information 
for non-commercial purposes. In an academic environment, for instance. Open Educational Resources (OER) are 
some benefits derived from CC license. This is because these teaching, learning, and research materials in any 
medium permit the free use and re-purposing by others. As a result, in regard to educators, finding, sharing, 
combining, and customizing text-books, courses, and lesson plans could be done freely and easily (Education n.d.). 

Notwithstanding, it is important to understand that CC license is not created to subvert the copyright system. 
Many people have the misconception that they have to give up their copyright when CC license is applied. This is not 
true. The authors of such works need not to give up their copyright when they use CC license. In fact, most of CC 
license criteria of permissions to the public are certainly based on the copyright. It is likely to comprehend that CC 
license just gives more freedom to the users and in the meantime controls how the information will be used. In UK, 
for example, the Open University launches an initiative entitled “OpenLeam” to “reduce the costs of course 
development for everyone in education and improve quality by openly sharing quality-assured educational materials” 
(OER Case Studies 2013). It makes use of CC BY-NC-SA, which allows others to remix, transform, and build upon a 
work non-commercially, as long as the work is credited and the new creations are licensed under the identical terms. 
(About the Licenses n.d.) 

The benefits of using CC license for individuals are that they can customize the rights they would like to give, and 
also still have control over their works. By contrast, a regular standard copyright cannot do so. Under the realm of 
copyright legislation, users have to stick to the copyright legislation no matter what they like or not. Authors, creators, 
or right holders have to compromise on several aspects of distribution when they use copyright. Many stakeholders 
acknowledge that CC license liberates them from all such things (Elkin-Koren 2005). Still, the works created for CC 
license are definitely protected under copyright law as such works are expressed into fixative and tangible forms. 
Thus, in order to use CC license, creators need to have a copyright. Their copyright can then be modified to suit their 
needs by CC license (Frequently Asked Questions 2014). 

Creative works are, indeed, copyrighted. Copyright law protects original works of authorship, and CC licensing 
scheme does not change this. However, it changes the pervasiveness of copyright. Licensing copyrighted materials 
used to be the domain of corporations. Individual creators were always the owners of their creative works, and works 
which were not intended for commercial use remained the sole property of the authors even after they were made 
available to the public online. Many works were posted online without any restrictions, on the implicit presumption 
that re-use was permissible for non-commercial purposes. This thriving environment of information, produced and 
shared by peers, has driven the Internet to its colossal success at present (Guibault & Hugenholtz 2006). Individuals 
never bothered to assert their rights or engage in licensing. Licensing was either too complicated or too expensive. On 
the whole, individuals did not expect any revenues from sharing their creative works, and normally avoided the legal 
cost of licensing. By reducing the cost of licensing, CC license makes licensing more accessible to individuals, 
thereby strengthening the hold of copyright in our everyday life. It seemingly empowers individuals with legal powers 
that were once available only to industry. It makes copyright accessible to all. Now that individual authors are not only 
aware of the proprietary regime but are also armed with an efficient mechanism to execute their intellectual property 
right, they may use it to set limits on the exploitation of their works. 

The metaphor of property is rather powerful. Intellectual property, however, is not merely a metaphor (Guibault & 
Hugenholtz 2006). It constitutes an effective legal mechanism that allows exclusion. The need to secure permission 
prior to the use of any creative works is the main barrier for sharing and collaborating among individual creators. It is 
the main cause of the transaction cost that CC seeks to reduce. 

As stated above, there are some concerns about the replacement of copyright by the adoption of CC license. 
According to the research’s analysis, it can be confident to affirm that CC license is not created to subvert copyright 
norm. Instead, it can work in parallel with the copyright law. As the understanding of the licenses spreads, greater 
numbers of content users, particularly educators, will specifically seek out materials that are covered by a CC license. 
A renewed sense of openness - rather than restrictiveness - will support new educational efforts, particularly in 
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developing countries, that cross institutional and national boundaries. Educational content will be shared freely, with 
programs and degrees based on learning objects and resources from multiple sources. Globally, both developed and 
developing countries are adopting CC license for education, ranging from Argentina to the USA (OER Case Studies 
2013). In addition, CC license can still be a useful tool to encourage people both right holders and users to publish and 
make use of copyright respectively via the Internet. However, it should also be kept in mind that the good system of 
CC must be consistent with the purpose of copyright law. This means CC license must be adopted to support the 
knowledge acquisition via electronic media and meanwhile it must maintain the value of copyright norm by 
broadening the means to access copyrighted materials transformed in digital devices. 


4. CONCLUSION 

The emergence of digital world has brought with it collaboration on a global scale. In today’s age of digital 
technology, including the sharing and collaborative environment of Web 2.0, arguments about copyright issues, 
especially copyright infringement, emerge. As CC license enables users to create and share creativity as they choose, 
it can, therefore, be regarded as a fascinating tool in a legal aspect that can benefit many, involving educators. It is 
undoubtedly used as a popular alternative to the “all rights reserved” copyright license that we are all familiar with. 
Nonetheless, it is important to note that CC does not replace copyright. Those who have chosen to use CC license still 
own the copyright of their works, they are simply licensing its use through CC license. According to the analysis, this 
research also proposes some interesting recommendations to uphold the great benefits of CC license to copyright 
norm as follows: 

1. CC license should not diminish fair use doctrine. Fair use already allows people to use copyrighted works. 
However, under the fair use doctrine, it is required to consider some factors to determine what is “fair”. In its most 
general sense, fair use of a copyrighted work is any copying of copyrighted materials subject to some limitations for 
some fair purposes, such as comment, criticism, news reporting, and scholarship (U.S. Copyright Office 2012). On 
the contrary, there is no need to consider any factors to make use of the copyrighted works through the adoption of 
CC license. This does not mean CC license is trying to diminish the value of fair use doctrine. Instead, it helps to 
broaden the accessibility of copyrighted materials in the digital era. It allows the free distribution of creators’ works 
for non-commercial purposes. Allowing broad noncommercial use of otherwise protected materials significantly 
expands the pool of resources educators can use in teaching and learning. Therefore, it is introduced to broaden the 
scope of fair use doctrine, which may be called as “fair use plus”. Under copyright law, nobody is allowed to 
reproduce copyrighted works without the right holders’ permission but with CC license, stakeholders - both creators 
and users - can keep hold of the copyright and still make money on the works, while allowing the public to spread the 
words widely. 

2. License terms of CC license should be transparent. In order to cut down the complication, high costs, and 
time consuming of traditional copyright license, CC license should set out a transparent range of licensed terms that 
allow people to state just how far they are prepared to share or give away for free their outputs. It is agreeable that 
each CC license comes with the same baseline user rights and restrictions. These allow the materials to be copied, 
distributed and reused, at a minimum in its current form, for non-commercial purposes, and as long as the original 
creators are credited. Once a contract is in place, the terms and conditions of the contract prevail over copyright law 
because a contract is considered to be a private bargain between private actors, whom are presumed to have equal 
bargaining power to negotiate the terms and conditions of the contract. To assist copyright law regarding its condition 
on licensing mechanism, the substantiality of CC license is to extend individuals’ opportunity to adapt the license to 
their needs regardless payment as there are many choices available. Some familiar CC license choices are CC BY, 
which allows others to distribute, remix, transform, and build upon a work, even for commercial purposes, as long as 
the work is credited for the original creation; CC BY-NC, which is identical to CC-BY but allows others to make use 
of this term under non-commercial purposes; and CC-BY-NC-SA, which resembles CC BY-NC unless others 
license their new creations under the identical terms (About the Licenses n.d.). 

3. Moral rights should be pondered within the adoption of CC license. The focus of moral rights is not on 
corporation or commerce. It is to offer legal protection for an author’s right to be acknowledged as the creator of his or 
her own work. To an extent, moral rights can also protect the integrity of creative works, empowering authors and 
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artists to protest damaging or demeaning treatment of their works. Indeed, the approach of moral rights to creative 
works is the foundation of the CC license, by means of which, a work is effectively released to the public. This form 
of publication ensures that the works are available for use without copyright restrictions, but this is not without 
conditions: attribution and integrity are protected, to varying degrees, and in various guises, as authors decide. The 
provision of CC license closely approximates the legislative provisions of moral rights. Also, author’s moral rights are 
in no way affected by CC license. Instead, CC license prevents anyone re-using the work from doing so for 
commercial purposes. Regarding perspective of moral rights, the third license, which adds a “no derivatives” principle 
to the attribution license, is noteworthy too. It provides that no person should make a derivative work based on the 
original work. This means this license allows for redistribution, commercial and non-commercial, as long as it is 
passed along with credit to original creators. 

4. CC license must be an optional tool to solve copyright conflict. Since CC license is not anti -copyright per se, 
but argues for use of more flexible and open copyright licenses within existing copyright law, it actually obscures the 
real copyright issues we face currently, and keeps people to settle on the proper parameters of digital information use, 
access, retrieval, and preservation in the 21st century. However, it should play as an optional tool to resolve a conflict 
surrounding copyright. In the best case scenario, with a balanced and effective law that serves citizens and corporate 
owners equally well, a CC license is unnecessary. This is because a good copyright system can already provide the 
expansion of protection as well as the expansion of private use, exceptions to and limitations on copyright, which is an 
essential means of striking the right balance. 
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ABSTRACT 

The paper selects over 20 online courses and analyses the subjects, organization, the way to show the content of the 
courses, the use of media, and design of the teaching in the case study of Chinese popular MOOC platform. On this basis, 
the paper summarizes the principles of curriculum design and design models in MOOC environment, such as practical 
course content, minimized but continuous course structure, simplified course content, the effective combination of the 
media with the content, leamers-centered, and stressing social construction. The author is not only a member of the 
curriculum development team of MOOC, but also experiences the online courses as a learner, and tries to summarize the 
principles of curriculum design and design models after analyzing the design elements with the case study method. 


KEYWORDS 

Massive open online courses, Curriculum design, Knowledge Construction, Cognitive load theory. 


1. INTRODUCTION 

MOOC has been opened in a number of universities, institutions and organizations. In the year 201 3, some 
top universities in China such as Tsinghua University and Peking University formally proclaimed to join 
edX, which is one of the three popular MOOC platforms. In October the same year, Tsinghua University 
formally opened the MOOC platform Xuetangx.com, thus making its curriculums available all over the 
world. As a member of the curriculum development team of Chinese MOOC, the author herself also 
experiences the online courses as a learner, and tries to summarize the principles of curriculum design and 
design models after analyzing the design elements with the case study method. Finally she hopes that this 
paper can be useful for the following curriculum development. 


2. CASE STUDY 

Over 20 online courses are selected and discussed about the advantages and disadvantages of their design. 
The selection of the subjects, organization, the way to show the content of the courses, the use of media, and 
design of the teaching are analyzed in details in the case study as follows. 

2.1 The Selection of the Courses and Their Organization 

According to the statistics, there are totally 147 online courses on the Xuetangx.com platform, including 
computer, cloud computation, entrepreneurial management, data science, social science and thinking training, 
etc. Among all these courses, over 20 subjects which are developed by Tsinghua University are selected for 
us to analyze. 

Most online courses on the Xuetangx.com are published weekly in each semester. The courses are 
introduced by chapter or topic. Every unit is showed in a short video which lasts 1 to 15 minutes. 
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Figurel shows the contents of the subject Principles of Electric Circuits, which is organized chapter by 
chapter. Figure2 is the resource list of the subject Principles of Electric Circuits, in which 99 lessons are 
arranged in 16 weeks. In addition, 198 tests, 109 assignments and 240 videos which last at most 12 minutes 
are also included. 
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Figure 1. Contents of Principles of Electric Circuits Figure 2. Resource List of Principles of Electric Circuits 

As we can see, every video features only one topic, short time and brevity, thus contributing to the 
concentration of the learners. 



Figure 3. Knowledge Structure of Principles of Electric Circuits 

Figure 3 is about the knowledge structure of the course Principles of Electric Circuits. The figure shows 
the hierarchical relationship between the knowledge. Besides, some pieces of the videos are arranged in 
accordance with the order of the knowledge. 

2.2 The Way to Show the Content by Multimedia 

The beginning video of every course is usually about the general idea of the course. It is designed to be 
dynamic in order to attract the students. 

History of Chinese Architecture has the unanimous style. As we can see in the figured, the start of the 
PPT, the explanation of the teacher and what is shown in the video are unified aesthetically and clearly show 
the style of Chinese architecture. 

Figure5 is a picture of the course Principles of Electric Circuits which combines various media such as 
blackboard, PPT, experiments, thus solving the problems of most science and engineering teachers. 

As shown in figure 6, the simple vivid animation stories are used in the course Financial Analysis and 
Decision Making, which is effective to explain the selling and purchasing process and their relationship. 



Figure 4. The course History of Chinese Architecture 
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Figure 5. The course Principles of Electric Circuits 



Figure 6. Video of Financial Analysis and Decision Making 


2.3 Design of Teaching 

2.3.1 Making up a Virtual Learning Environment 

Most online courses are divided into semesters on the Internet. Instead of putting all the course videos on the 
Internet at one time, we upload the new course video weekly. Thus we make up a learning environment 
where there are semesters, homework, tests, examinations, course credits. The learning environment enables 
students to have a sense of belongings. 

2.3.2 Interaction 

Figure7 is about the drawing and writing on the PPT in the course Principles of Electric Circuits, which can 
attract the students to follow the teacher’s ideas to study. Besides, it is much more attractive than just writing 
knowledge on the blackboard. 

Figure 8 is about the quiz between videos. Only by finishing the quiz can the students continue to study, 
which plays the same role as questioning in the class. 
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Figure7. Principles of Electric Circuits- Drawing and Writing 

2.3.3 Communication 



Figure8. Electric Circuits-Test between Videos 


The Xuetangx.com platform provides communication function. Students can introduce themselves to others. 
Cultural communications among students from different places can also be done. They can share the study 
materials and information by posting to the platform. Through the online platform, the teacher may ask the 
students to define a vocabulary and students themselves can modify each other’s definitions of the 
vocabulary. As soon as one student raises a question, other students will begin to discuss it spontaneously 
before the teacher answers it. 
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2.3.4 Evaluation 

The Xuetangx.com platform also has the function of self-evaluation on which the students can see their 
course progress and scores. And that is an inspiration for the students. 



Figure 9. Communications on the Online Platform 
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Figure 10. Principles of Electric Circuits-Study Evaluation 


In conclusion, every course has the design of teaching, including a virtual learning environment, the 
interaction, the forum of each course, the function of self-evaluation and resources. However, we can see few 
online courses which involve in the cooperative learning and research. 


3. PRINCIPLES OF CURRICULUM DESIGN 

Basing on the analysis and summary of the design of teaching after our experiencing of the online courses, 
we draw a conclusion of the design principles of the curriculums in the MOOC environment. 

3.1 Practical Course Content 

The investigation of the learners in e-learning courses shows that the biggest cause of their stopping the 
learning is that they are not interested in the content. Research shows that examples or problems from our 
real life are helpful to inspire or maintain the students’ interest when learning. The design of MOOC should 
emphasize on inspiring and maintaining the interests of students. So the key is the practical course content. 

3.2 Minimized but Continuous Course Structure 

The cognitive resource of the learner is limited according to the cognitive load theory. Once the information 
is overload, the meaning-construction of the knowledge will be influenced (Rohani Ahmad Tarmizi & John 
Sweller, 1988). Researchers such as Gerhard Ross from the European brain lab had also proved that minimal 
content and frequent repetition are the best for study from a neurobiological perspective. 

We should follow the principle of modularization, short time, refined knowledge granularity and focusing 
on the important knowledge. In other words, the course content should be divided into small modules 
according with the teaching goal, each of which should only have one knowledge point and last 5 to 10 
minutes. However, we don’t suggest random division ignoring the systematicness of the course. 
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3.3 Simplified Course Content 

Good cases of the online courses can best explain and simplify the content. They are usually characterized by 
the simple writing performance, graphs and animation stories which can show the relationships. 

As to the way of course content presenting, it should follow the cognitive load theory so that the inner 
cognitive load of the students can be lessened. Therefore, the following suggestions can be adopted: reducing 
the irrelative information on the screen, explaining the abstract content with graphs. Meanwhile, we should 
also stress the improvement of learner’s relative cognitive load. Marking the key content or setting reminders 
can promote the efficiency of the learners. 

3.4 The Effective Combination of the Media with the Content 

Numerous studies show that a combination of various media is conducive to the construction of knowledge. 
For example, with long-term research, Richard.E.Mayer concluded that it was more efficient for students to 
be shown with both words and pictures than just the words in the class. Only by being shown both the words 
and pictures can students form a module of them and establish certain relationship between them. Finally the 
meaning study is realized. 

Hence, a good design of the course should be, basing on what is to be presented, the appropriate 
combination of the media. 

3.5 Learners- Centered 

In the MOOC environment, learners are the subjects who choose what they learn and control their own 
progress. Virtual learning environment should be built to support learners with different learning modes. 

The MOOC environment should meet the following requirements: first, the knowledge should be 
arranged step by step so that learners can do independent learning and complete their knowledge construction 
smoothly. Second, diversified resources should be provided to meet the individual needs of the learners. 
Third, platforms for the communications should be offered to support collaborative learning. Last, automatic 
mechanism for evaluation of students’ study together with the homework, tests and examinations should be 
given to students, aiming at a timely feedback for students’ study. 

3.6 Stressing Social Construction 

In the virtual classes, we should attach great importance to the guidance of students’ communication, the 
efficient problem-solving process for the students, and the providing of various network tools for students to 
communicate and share knowledge. If so, the students can get their knowledge strengthened and learn 
something new through the Internet. 


4. CONCLUSION AND FUTURE RESEARCH 

The curriculum design and development in the MOOC environment is meaningful. The author hopes that this 
paper can be useful for the future development of the MOOC. With a direct analysis of the MOOC, this paper 
might be limited in the design principles of the curriculums, which need to be revised and improved in the 
future. 

The future research will focus on the modes of the curriculums in the MOOC environment, aiming at 
providing a better way for teachers to develop MOOC curriculums and improving learners’ efficiency. 
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ABSTRACT 

Tourism sector is a key element of the Maltese economy. Therefore, having an appropriately trained workforce becomes 
a critical factor in ensuring that tourism continues to flourish. In order to avoid skills mismatch and similar problems, it is 
extremely important to identify the key stakeholders and encourage them to participate in the process of curriculum 
change in Maltese Tourism Higher Education (HE). A successful stakeholder analysis will determine their degree of 
influence and as a result devise appropriate mechanisms for their involvement in curricular reform. Failure to do so often 
results in an unsuccessful reform project. 


KEYWORDS 

Stakeholder Analysis, Higher Education 


1. INTRODUCTION 

The current socio-economic scenario presents challenges ranging from aspects of globalisation, work-related 
migration and in particular the need to be able to engage in an environment that no longer has a 
homogeneous culture but rather a mix of cultures. This is a challenge that Higher Education (HE) institutions 
need to redress in the sense that the 21 sl century student has to be exposed and adequately prepared to operate 
in such an environment. The OECD (2012), the European Commission (2013) and other constituted bodies 
are taking diverse research initiatives that are looking at this aspect of a multicultural environment and how 
to effectively take on the advantages that may be brought by such an environment, yet at the same time 
mitigating the possible negative consequences arising from culture-based misinterpretation of human 
behaviour. 

The counter argument is that HE graduates do not have the ‘right’ cultural skill set for today’s economic 
scenario. The gap in students’ skill-set is a cause for concern for the graduates, academics and employers 
(Times of Malta, 2011) in Malta. Studies from different countries share similar concerns (Constable & 
Touloumakos, 2009). 

This investigation forms part an ongoing research process that is investigating the assessment and 
accreditation of culture-related ‘soft skills’ through the implementation of an e-learning system within the 
Institute of Tourism, Travel and Culture (ITTC) at the University of Malta (UoM). The research focuses on 
undergraduate students reading for the B.A. Tourism Studies who, upon graduation will most probably take 
up employment in the Maltese Tourism sector. This paper looks at the need of carrying out an adequate 
stakeholder analysis exercise to ensure a successful project outcome. 
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2. THE MALTESE SCENARIO 

The importance of the Maltese Tourism and Hospitality sector may be seen from its contribution to the 
Maltese economy. With over 25%* of the country’s Gross Domestic Product (GDP) it is by far the most 
significant sector which is well above the EU average of around 8%. (Azzopardi, 2011). The number of 
persons employed directly in hotels and restaurants alone on a permanent basis, amounts to approximately 
6% of the entire country’s workforce (NSO, 2013). 

At the same time, (NSO, 2013), visitors to Malta are diverse in terms or country of origin, ethnicity, 
religion, cuisine and scope of the visit itself (sun & sea, heritage, culture, leisure activities, etc.), than before. 
This has posed a series of challenges in that various operators in tourism, hospitality, culture and heritage 
management, etc., must be aware of the multicultural environment in which they are now operating. 
Educators operating in tourism, travel and culture also need to face up this challenge. 

In order to ensure a successful implementation, the key stakeholders need to be identified and involved in 
the process; in this case that of curricular development including the use of e-learning technologies. Bryson 
(2004) refers to Nutt’s (2002), study of 400 strategic decisions that suggest that half of them failed (not 
implemented or implemented with poor results) because the decision makers did not seek information from 
and at the same time redress the needs of the stakeholders involved. Chapleo and Simms (2010) refer to the 
importance of identifying stakeholders their influence as being of fundamental importance to policy-makers. 
Failure to do so often results in poor performance if not a complete disaster (Bryson, 2004). 


3. STAKEHOLDER IDENTIFICATION AND ANALYSIS 

Freeman (1994) defines stakeholders as groups or individuals that may affect or are themselves affected by 
the achievement of an organisation’s objectives. Mead and Andrews (2011) discuss stakeholders as being 
fundamental towards successfully implementing a strategy. They quote David’s (1993) definition as 
individuals and groups of persons who have a special stake or claim on the company. 

Marie (2013) makes a direct reference to the role stakeholders may play in affecting HE. Getting to know 
the stakeholders’ perspective is an essential element in order to ensure that universities are able to fulfil their 
mission. 

Bryson (2004) illustrates various techniques that can be used to carry out a comprehensive stakeholder 
analysis exercise. He suggests that as a starting point, one should employ a grid and map the interest and 
power or influence of each stakeholder group on a quadrant (Bryson 1995: 71 -5). 

Bryson (2004) goes on to indicate Mitchell’s et. al., (1997) work that, in order to be able to position the 
stakeholders along the power/influence quadrant one has to be able to; 

1 . specify how each stakeholder may influence the organisation 

2. determine what the organisation needs from each stakeholder 

3. rank stakeholders according to their importance to the organisation (by considering stakeholder’s 
power, legitimacy and attention seeking) (Mitchell et. al, 1997) 

3.1 Identifying ITTC Stakeholders 

A process involving the detailed analysis of the documentation governing the ITTC (2010) was carried out in 
order to identify the main stakeholders that may have an interest and possibly have an influence on the ITTC 
HE curriculum. This first outcome was to look at all potential stakeholders and classify them in two broad 
categories, external and internal (Lewis, 2006). 

For the purposes of this study, external Stakeholders would be those not ‘residing’ within the UoM and 
the ITTC. The current governance structure of the UoM (2012), makes it unlikely that they are able to have 
a direct involvement the development of the ITTC’s strategies and policies, including curriculum 
development (ITTC, 2010). They would however, be affected by the decisions taken within the ITTC. 
These stakeholders were: 


It is thought that the overall contribution is closer to 40% based upon estimates carried out by the Malta Hotels and Restaurants 
Association 
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1. Malta Tourism Authority (MTA) - the principal government agency involved with developing 
Malta’s tourism strategies, policy planning, legislation and enforcement, etc. It is the Maltese 
government’s official representation for the tourism sector. 

2. Heritage Malta - the government agency entrusted with the running and management of various 
heritage sites. 

3. Malta Hotels and Restaurants Association (MHRA), representing the vast majority of 
operators/employers in the Maltese tourism and hospitality sector. 

4. Institute of Tourism Studies (ITS) - a state managed vocational college in the field of tourism and 
hospitality. Students from ITS on completion of Higher Diploma courses may enrol for an 
undergraduate degree course at the ITTC. 

5. The tourists visiting the Maltese Islands. 

On the other hand, internal stakeholders are found within the UoM. Although not all be directly involved 
with the planning and development of ITTC curriculum, their influence and power welding is determined by 
the administrative and managerial roles that they may occupy within the ITTC and other UoM entities. They 
are thus in a position to determine whether a proposed curricular change would be, approved, amended and 
ultimately implemented. The following internal stakeholders were identified: 

1 . ITTC Director 

2. ITTC Academic Staff 

3. ITTC Board Members 

4. Other UoM Academic Staff 

5. ITTC Students 

6. UoM IT Services 


4. FINDINGS 

The official documentation related to the governance of the ITTC (ITTC, 2010) within the framework of the 
University of Malta’s governance (UoM, 2013) was analysed to determine the actual influence that diverse 
stakeholders have and the power that they may wield in curriculum development and management. Other 
documentation, internal to ITTC, was reviewed. These included documentation related to the employability 
of ITTC graduates, ITTC’s developmental strategy, annual reports, and ITTC’s journal, dissertations, 
examinations and post-graduate committees. It is significant to note that the ITTC director either chairs or 
co-chairs each of the above-mentioned committees and the appointment of the other members lies within the 
Directors remit, albeit subject to the ITTC board approval. Another finding was that the almost all the 
committee members were in fact academic staff members of the ITTC. In some, isolated cases, other, non 
ITTC UoM academic staff members were found. As with the previous group, these were also subject to 
approval by the ITTC board. 

This documentation review indicates that the ITTC Director has a considerable amount of power and 
within the ITTC. However, this is counterbalanced by the ITTC board, whose composition does include other 
stakeholder categories that do have considerable interest in the ITTC itself. The terms of reference of the 
ITTC specify clearly (ITTC, 2010) that it is this board that is entrusted with the functioning of the Institute 
and setting of policies. The ITTC board is made of: 

1 . Chairman: Rector or his delegate; 

2. Vice-Chairman: Director of the Institute; 

3. a person appointed by the Council of the University; 

4. a person appointed by the Senate of the University; 

5. up to two representatives of the academic staff lecturing at the Institute; 

6. up to two representatives of the students registered with the Institute; 

7. one scholar of repute to be recommended by the Board and approved by Senate; 

8. one person actively involved in the field of tourism, travel and culture nominated by the Board 
and approved by Senate; 

9. one person nominated by the Board of Governors of the Institute for Tourism Studies and 
approved by Senate. 

(ITTC, 2010) 
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In order to gain further understanding of the power and influence that internal stakeholders have over the 
ITTC, two interviews were carried out with two out of a total of seven full time academic staff members of 
the ITTC. The first member is a very recent addition to the ITTC academic team. Since the mid- 1 970 ’s he 
has held various very high profile tourism management and consultancy positions and has over 30 years of 
experience within the Maltese tourism sector. The second member is an academic by profession from the mid 
1990’s occupying various academic and administrative roles. He was on the University Council which is 
responsible for the general administration of the University (UOM, 2013) and currently a member of the 
ITTC board which as indicated earlier directs the academic tasks of the Institute, as it determines the studies, 
teaching and research within the Institute. 

The outcome of these interviews was that when it comes to ITTC curriculum design and development, the 
key stakeholders within the ITTC are the ITTC board and the ITTC Director who ultimately sets the policies. 
Academic staff are do have some influence and power due to the positions that they may occupy on the ITTC 
subcommittees, but the ultimate decision making lies within the ITTC board. ITTC students are represented 
on some of the subcommittees and therefore may bring their views forward; their representation is always of 
minority. Thus, while some aspects of the ITTC curriculum design would be of interest, their degree of 
influence is limited. 

However, when the issue of utilising ICT/e-learning for teaching and learning within the ITTC, it was 
pointed out that any decisions taken are subject to approval by the UoM IT services. A simple analogy was 
made by referring to the current ITTC website, which is hosted on the UoM IT services servers. The website 
template and layout of the ITTC website was set by the UoM IT services. Amendments are regularly made 
by the ITTC staff, but all the changes are subject to approval by the IT services supervisory staff. Similarly, 
any e-learning related initiative is subject to their approval. Hence although not on the ITTC board, the UoM 
IT services unit has significant influence over curricular development that involves ICT/e-learning. 


5. CONCLUSIONS AND FUTURE WORK 

This exercise served to identify and determine the degree of influence and power that potential tourism 
education stakeholders may have had in determining the ITTC e-learning curriculum. The ITTC Director 
and ITTC board are highly influential stakeholders. Their status set by the ITTC governance rules gives 
them both influence and a high degree of power as they are the ones determining the policies (including 
curricular development) and implementing the objectives of the institute. For any reform to be successful it 
is imperative to have the ITTC director as the main champion as s/he will be sponsoring these reforms with 
the ITTC board. However, another significant stakeholder would be the UoM IT services section as they 
determine the mode of engagement with the current UoM IT systems, for any IT -related initiative including 
e-learning. Being able to engage successfully with these stakeholders becomes a critical factor for the 
successful implementation of proposed e-learning related curricular changes. 

One possible way being considered for engaging with the various stakeholders is through the use of a Soft 
Systems Methodology (SSM) (Checkland & Scholes, 2004). SSM has been used in order to illustrate the 
complexity of innovation within an educational setting (Cox, 2010) and get a better view of the stakeholders’ 
position. This should help in devising effective forms of stakeholder engagement in order to ensure the 
successful implementation of the proposed e-learning curricular reform from a technical point of view and at 
the same time, addressing the culture skills gap indicated earlier. In practical terms, providing university 
graduates with the right set of skills and knowledge enabling them to engage successfully in today’s 
globalised, multicultural environment, contribute towards socio-economic growth and fulfil their own 
personal development. 
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ABSTRACT 

Currently a large number of college students belong to social networks and spend several hours a week on them. Some 
sectors of society, like parents and teachers, are concerned about the negative impact on their academic work and in their 
personal lives. However, because the potential positive impacts have not been explored enough, this research addresses 
the question: What are the characteristics of online social networks that could facilitate the development of college 
students skills’? The research was qualitative and was conducted in a private institution of higher education. Twenty-one 
college students and thirteen college teachers participated. The results show that the characteristics of social networks 
that favor the development of ICT skills are primarily: 1) environment conducive to learning where students can learn by 
playing, racing, linking and imitating; 2) rich environment in technology tools, where there are primarily face tools, 
external tools executed outside the social network, internal tools, and external tools executed within the social network; 
3) appropriate environment for the exchange of information, where communication can be: one to one, one to many, one 
to all contacts and one to all followers. 


KEYWORDS 

Online social networks, ICT skills, virtual ethnography. 


1. INTRODUCTION 

Information and communication technology (ICT) development has been enabling communication between 
people to become simpler and faster. Internet penetrations, emergence of new electronic devices, online 
social networks, and overall improvements in the area of computing have completely transformed the way we 
live and communicate. However, the complex environment of today's society requires the development of 
digital competences. These competences include the skill set that allows managing information effectively 
and properly utilizing ICT (Wallis, 2005). 

For today’s college students, developing computer skills is essential for their everyday interaction with 
the world; not only in education, but in any aspect of their lives. Oblinger and Oblinger (2005) explain that, 
for the Net Generation, technology is not new; they are not surprised by the advances in computer, Internet, 
and other online resources, because these already existed when they were born. 

Online social networks are part of these available tools. While some studies have analyzed the possible 
negative impacts of the use of online social networks (Silkoset, 2013), it can be assumed that using them will 
improve computer skills. Online social networks are information systems that allow interaction of millions of 
people distributed around the world (Kazienko & Musial, 2006). Facebook, Myspace, Twitter, Linkedin are 
just popular examples of a broad universe of social networking tools that are available to the public through 
the Internet. 

Facebook is the platform selected for the purposes of this study. This selection was made not only 
because of the overall number of users but for its implications in education. In the past many studies have 
shown the high participation of college students in the platform, usually around 90% are active members 
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(Hargittai, 2008, Smith & Caruso, 2010); studies also identify the same frequency of daily use by this 
population (Christofides, Muise, & Desmarais, 2009; Pempek, Yermolayeva, & Calvert, 2009). In addition, 
there is an emerging theoretical development in identifying applications of Facebook in learning 
environments (Mazman & Usluel, 2010; Barczyk & Duncan, 2013) that generally reflects the receptivity of 
students of the use of this technology in the classroom. 

In order to take advantage of the online social networks it is important to identify the characteristics of 
online social networks that could facilitate the development of college students’ digital skills. Given this 
situation, the question that this study addresses is: What are the characteristics of the environment in online 
social networks that could develop computer skills? 


2. METHOD 

This research is qualitative, and is divided into two separate studies which are not dependent on each other. 
According to Creswell (2008), one of the advantages of qualitative research is the ability to achieve a holistic 
understanding of the phenomenon under investigation. This is why it was decided to continue the naturalistic 
paradigm as it provides an overview of the experiences, values and beliefs of the participants. In this type of 
research theory emerges and develops from the data (Lincoln & Guba 1985). 

The first study included thirteen teachers experienced in e-learning and use of Web 2.0 tools. We used the 
focus group and the formal interview as tools for data collection. Data analysis was performed with the 
method of constant comparison of Lincoln and Guba (1985). The second study included twenty-one 
university students. It consisted of a virtual ethnographic design, in which online participant observation was 
monitored, using Facebook and informal interviews, during a semester. To analyze the data obtained from 
this observation the Spradley (1980) methodology was followed. The method of constant comparison of 
Lincoln and Guba (1985) was used to analyze the information from the interviews. 


3. RESULTS 

Overall results show that online social networks have the potential to encourage the development of skills 
through its ludic environment, rich in technological tools and plenty of spaces that promote information 
exchange. 

3.1 Favorable Learning Environment 

The development of competences involves learning, and social networks provide a learning environment 
where students can play, imitate linking and even compete; 

• Learning by playing. In this research, teachers and students considered that the relaxed and ludic 
environment of these tools is useful for developing skills for using new technologies. According to one 
participant (teacher): "People normally access a social network, such as Facebook, to socialize. In fact, 
participating in Facebook for entertainment causes changes in personal attitudes”. Even if it seems like 
you are just playing, "you are actually acquiring some useful skills, in a natural way. Although it is 
entertainment, these skills will help you in the future". 

• Learning by imitating. This environment fosters the exchange of information, encourages users to see 
what others are doing, and it creates a kind of imitative behavior. During research, for example, several 
practices were initiated and eventually were replaced by others. Although many of these “trends” are 
mere amusement, some of these help develop skills that can later be used in completely different areas. 
For example, the logic of Notes on Facebook is very similar to that of Blogs. 

• Learning by linking. In general college students not only learn by imitating, but they also learn by 
linking. Students enter online social networks for many reasons, but none of the students said that 
learning was one of them. However, a user can identify elements of amusement in his network, as 
well as recognize its utility in academic terms. An example of this is when certain functions of 
Facebook can be used in other computer systems in academic or professional areas. 
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• Learning by competing. In this environment, students can learn to compete. In fact, competing with the 
popularity of a friend or member of the network promotes interest in "making" improving their space 
(profile). Users like to experiment with new tools, using their creativity. According to one professor: 
"these tools are used as a kind of competition because an important part in these networks is the profile, 
and how sophisticated it is... that has helped people to focus more on certain computer competences". 

3.2 ICT-rich Environment 

Facebook users use different tools in their social practice. The large majority of the tools that they use are 
supplied by the same platform, but not all. During the observation, four types of tools were identified: 
presential tools, digital tools external to the network, tools provided by the same online social network and 
external tools executed inside the social network. 

• Presential tools. This category includes diverse tools like digital cameras and mobile phones. These 
tools, although typically not observed in the interaction on the platform, are very important in the culture 
studied. These devices produce much of the information shared within the online social network, through 
photos and videos. Furthermore, some of these mobile phones also enable access to the platform 
immediately, through mobile technology. Therefore, users of online social networks share information 
asynchronously. 

• Digital tools external to the network. These are tools very important because users use it for their content 
preparation or for finding interesting information. For example: photo and video editors to “prepare” the 
content; video sharing sites like Youtube , to share something funny; blogs or webs to share news. 

• Tools provided by the same online social network. These include different types: wall, mailbox, chat, 
video calling, photos, video, links, events, birthday. 

• External applications executed within the social network. External applications are developed by 
individuals or external companies, but are still valid for use in the social network environment. Many of 
these apps are games or recreational applications enabling users to play. These applications also generate 
the interaction network, and some of them allow simultaneous joint participation. 

3.3 Information-exchange Favorable Environment 

During research, four different types of communication and information sharing options were identified: a) 
one to one exchange, b) one-many exchange, c) one- all contacts exchange a-d) one- all followers exchange: 

Exchange one to one. 

• Chat. The function of chat is to exchange information with a user privately. However, group chats can be 
set up where multiple users can sign up and interact in a conversation at the same time. It works 
basically like any synchronous messaging tool, but is designed for the same social network platform. 

• Private Message. This is the most direct and confidential way to, asynchronously, send information, or 
have a conversation with a user or group of users. Private message users can send information at any 
time of day, allowing users flexibility to answer. 

• Video call. This is the most direct way of engaging in dialogue, and includes the voice and image. It 
streamlines the communication process and is more personal, since people can actually see each other. 
So far, it does not allow for multiple users from different computers to interact simultaneously in a video 
call. 

Exchange of one to many 

• Groups. These are spaces that can be open, private or even hidden, depending on the configuration. All 
Facebook users are entitled to create these social spaces, which are commonly created to work around a 
topic. 

• Events. This is very similar to a group, unless the activity revolves around a particular event with 
organizer features, guests, date, place and time. This application allows you to create and communicate 
an event with detailed information about it. 
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Exchange of one to all your contacts 

• Profile's page. Contains the latest news concernig people that are of interest to the user and that are part 
of their social network. In this space, for example, the user receives information requests from people 
who want to be part of your social network, invitations to events and groups, birthday reminders and 
even promotions. 

Exchange one to all followers 

• Pages. Facebook pages are usually used by a person or entity when the group activity revolves around 
him/it. Pages do not require registration, so anyone who gets the address can reach the page without 
invitation. However, the page has to be created by a Facebook user. Pages have virtually the same spaces 
and tools as a Group. 

It is noteworthy that in addition to these internal applications of Facebook, there are several external 
applications: games, applications for music, apps to make surveys, and others. These tools involve various 
types of communication (one to one, one to many, one to all) depending on the nature of the particular 
application. To summarize. Figure 1 shows the characteristics of online social networks. 



Figure 1 . Environment properties of online social networks that favor the development of IT skills 


4. CONCLUSION 

Online social networks were not created for the development of competences. The business of online social 
networks does not seem to be in education but in socialization, as pointed out by McLaughlin and Davenport 
(2010). Users typically access these networks with the sole purpose of socializing, having fun, relaxing. 
Learning is not one of the most frequent reasons for accessing those sites. However, the process of 
socialization through these online social networks can encourage the development of computer skills. 

Given the characteristics of these social networking platforms, using them can facilitate the development 
of certain competences that may be helpful professionally in a knowledge-based society. The results of this 
research demonstrate that the particular situation of online social networks (the structural features of the 
platform of online social networking, the nature of college students, the activities they perform in the 
platform, and the tools) enable the use and development of certain competences. 

Participation in online social networks favors the development of skills in a relaxed atmosphere, with a 
largely playful approach which invites users to explore the many tools at their disposal. Online social 
networks are essentially spaces for moving data through multiple tools (Korzynski, 2013). These spaces of 
information transfer have the distinction that they concentrate a variety of tools in a single place, 
(Wattanasupachoke, 2011). These tools are often used separately: messaging tools, mailboxes, collaborative 
groups, forums, event management tools, calendars, contact directories, photograph management tools, video 
management tools, link management tools, Web pages and many more. 

The combination of a playful environment and access to numerous tools for collaboration and interaction 
at different levels creates an ideal environment for students to explore, without pressure, many of the tools 
available. This stimulates college students to engage in activities that are usually perceived as boring or 
complicated. For example, many of the LMS (Learning Management Systems) used in universities as well as 
collaborative tools used in the workplace, have similar tools to those offered on online social networks. 


297 




ISBN: 978-989-8704-08-5 © 2014 


However, it is important to note that not all college students use the same amount of tools while they are 
connected online. Thus, the development of skills does not occur to the same extent. In fact, it is important to 
point out that we are not saying that the use of online social networks will always have a positive impact on 
the development of skills. Like all tools, the outcome depends on how it is utilized. 
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ABSTRACT 

In laboratory courses, the assessment of exercises and assignments typically is treated as a simple, quantifiable approach. 
This approach however rarely includes qualitative factors, especially if the grading is being automatically performed by 
the system, and provides little to no feedback for the students to reflect on their work. The role of the laboratory in 
engineering education however is very critical and engineering students must possess knowledge that goes beyond mere 
theory, therefore a diverse and multidimensional assessment of laboratory students is a necessity. In this paper, an 
educational tool for laboratory students implementing automatic assessment based on artificial intelligence is being 
proposed, based on a pilot version that has been developed for MATLAB -related coursework. The highlights of the 
proposed tool is that it is based on a proven cognitive theory, it is easy to compile and or modify its contents, it is based 
on the same laboratory environment that the students are being trained in, it can include qualitative evaluation and it can 
save time that the educators would otherwise require to manually evaluate the exercises. 


KEYWORDS 

Text adaptation; interactive learning environment; student profiling; computer-assisted education; laboratory education. 


1. INTRODUCTION 

In laboratory courses, the assessment of exercises and assignments typically is treated as a simple, 
quantifiable approach. The student turns in an assignment and or exercise and the educator or system is 
expected to grade it. This approach however rarely includes qualitative factors, especially if the grading is 
being automatically performed by the system, and provides little to no feedback for the students to reflect on 
their work. This is especially true if the assessment is based on simple questions that have only one correct 
answer (Bush 1999), or if the assessment is only based on the end result, neglecting the efficiency of the 
solution (Carter et al. 2003). The role of the laboratory in engineering education however is very critical and 
engineering students must possess knowledge that goes beyond mere theory (Feisel and Rosa 2005), 
therefore a diverse and multidimensional assessment of laboratory students is a necessity. 

Educators can examine and grade the exercises either manually or by using automated tools. Automated 
tools significantly reduce the effort required for such a task, especially when there are a significant number of 
students; however, they are usually limited to a quantitative evaluation and cannot provide specialized 
feedback to the students (Douce et al. 2005). Depending on the laboratory course, they also may operate 
outside the actual tool/software that the students are being educated on, in an educational environment alien 
to the students, frequently treating the examination the same way as that of a theoretical subject (Ihantola et 
al. 2010). 

In this paper, we suggest the creation of an automatic laboratory course assessment tool that can provide 
personalized feedback to each student, allowing the students to discern their strength and weaknesses. The 
approach suggests that the tool should operate in the same environment that students have been using during 
their laboratory course. For this purpose, we chose to present an adapted MATLAB course, as it is the most 
widely used software in laboratory engineering education and covers a wide range of subjects (Harris et al. 
2002, Schmid and Ali 2000, de Magistris 2005). The approach presented in this paper is based on the text 
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comprehension theory of Denhiere & Baudet (Denhiere and Baudet 1992). This suggests that the texts and 
assessment questions of the tool should be adjusted into three different versions, R (Relational), M 
(Transformative) and T (Teleological). Details on the adaptation of material to meet the requirements of the 
applied comprehension theory can be found in (Samarakou et al. 2013b, Samarakou et al. 2013c). 


2. DEVELOPMENT OF THE AUTOMATIC ASSESSMENT TOOL 

The main objective of the content management system should be the easy, quick and user-friendly handling 
of both the educational resources and student data, in order to require little or no experience for anyone with 
basic computer skills to use it. As such, it is vital to simplify the system as much as possible by implementing 
widely known tools and resources. As MATLAB is a very flexible software, it is easy to develop an 
automated assessment tool that the students may use without having to result into another environment. The 
entire process can be introduced as a set of MATLAB ".m" files, programmed to initiate the assessment 
process once the student decides to access the software. The use of a spreadsheet file as a database for the 
educational material is also recommended; as MATLAB is capable of accessing spreadsheet files, it is very 
easy for the educator to provide the educational material via such a file, which may be very easily updated 
and or modified by virtually everyone with basic computer knowledge. Such a file can include everything 
that is required for the coverage of a specific topic, including the questions, technical texts and the data 
related to the performance of the students (Fig. 1). As such, it is easy for the educator to check the 
performance of each student and or automatically create a summary file with the performance of any number 
of students. 



Figure 1 . Example of a spreadsheet file used as a database for MATLAB 


2.1 Specifications and Requirements 

In order to develop an automatic MATLAB assessment tool based on the proposed cognitive theory, there are 
a number of requirements that need to be addressed. The requirements are both technical and educational and 
can be split into two groups, those intended for educators and those intended for students. 
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For educators, the requirements are: 

1 . The ability to create a virtually infinite number of new educational material 

2. The ability to input/modify new educational material (texts, questions, etc.) into suitable fields, 
without restrictions on their number, as well as the number of characters and answers to each 
question. 

3. The ability to designate the kind of text/question entered into the database, according to the cognitive 
theory. 

4. The ability to create activities with appropriate connection to the inserted educational material. 

5. The ability to create diagnostic tests for a single section with a selection, random or forced, from the 
pre -inserted questions 

6. Protection of the database file, so that it will not be modified by unauthorized users. 

7. The ability to share educational material between authorized authors/users. 

For students, the requirements are: 

1. The ability to register into the system (Name, Surname, ID Number) 

2. User-friendly interface 

3. The ability to choose any path after the diagnostic test, regardless of the received suggestion 

4. The short and easy explanation of the differences between the different texts and activities offered 
by the system (R, M and T type in our particular example) 

5. The ability to request additional assistance/feedback regarding a question and or text 

6. The ability to see the results of the taken tests 

2.2 The Student Environment 

Once the educator has set up the appropriate files, they can be either inserted into the workstations of a 
laboratory before the examination or provided to the students via an online portal. If the educational process 
is taking place into a controllable environment, such as a laboratory, the system can be programmed to 
deliver and or combine the assessment results of each student to the computer of the educator through the 
laboratory's computer network. If the files are given through an online portal, the files with the results will 
have to be sent manually back to the educator and they should be followed by appropriate instructions. It is 
thus vital for the file containing the results to be protected against unauthorized access, to prevent tampering 
before it is sent back to the educator. 

The MATLAB .m files should be programmed to be able to utilize a simple content database, which is a 
typical spreadsheet file in our example, and generate a file with the assessment results of each student. In 
order to do that, the program will ask the student of some basic identification data, such as the name, surname 
and student ID number. Based on the comprehension theory implemented for our example, there should first 
be a diagnostic test, assessing the prior knowledge of the student, followed by a recommendation to access 
further educational material. This educational material may be of either R, M or T type, depending on the 
cognitive profile of the student generated from the results of the first test (Samarakou et al. 2013c). 

Once the diagnostic test is completed, the student is provided feedback on particular strengths and 
weaknesses. The software will automatically assess on which type of cognition the student is the least 
competent on (R, M or T) and will offer feedback to the student, alongside with the suggestion to supply 
additional educational resources. It is vital to allow the student to choose resources of any type, regarding the 
results of the text, or even skip the additional material entirely and proceed to the final test. After the student 
completes the final test, the system should generate a file with the student's answers, profile, results and other 
data of relevance that may be recorded during the test, such as the time taken to answer each question and the 
number of times the student requested additional assistance from the software. Such data may be used to add 
a qualitative portion into the evaluation process (Samarakou et al. 2013a). After the completion of the 
educational process, the student should be clearly informed of his/her progress by the system in detail, which 
includes his/her detailed performance on every (diagnostic, feedback, final) stage of the process. 

The students should be using MATLAB during the entire process, working in the same environment that 
they were being taught during the course. The tool should provide questions and problems by automatically 
and possibly randomly extracting them from the content database file. These questions can range from typical 
"True - False" type to simple problems. It is also possible to add the ability for the student to request 
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additional help during the process. In our example, this ability has been implemented, as it is a recommended 
part of the cognitive theory that the proposed system is based on, and the student can request additional help 
at any time by simply typing a command. 

2.3 Evaluation Results 

A preliminary evaluation of the proposed system took place in the Informatics laboratory of the 
Technological Educational Institute of Athens, during the course "Introduction to MATLAB". A group of 40 
students has been asked to use a preliminary version of the tool and then provide feedback through a short 
questionnaire. The students are expected to take at least ten laboratory classes per course, accompanied by 
ten laboratory tests. The laboratory tests are normally given and assessed manually by the educators, but one 
test has been skipped and the students have been asked to use the proposed tool instead. The questionnaire 
and the results of the survey are displayed in Table 1. 


Table 1 . Survey Feedback 


Question 

Strongly 

Agree 

Agree 

Neither 
Agree 
or Disagree 

Disagree 

Strongly 

Disagree 

The tool was straightforward and easy to use 

80% 

15% 

5% 

0% 

0% 

The instructions were clear and helpful 

70% 

12.5% 

15% 

2.5% 

0% 

The tool has helped to improve your comprehension 
on the subject 

75% 

15% 

2.5% 

5% 

2.5% 

The time required to complete the process was 
reasonable 

40% 

40% 

15% 

2.5% 

2.5% 

You believe that the automatic assessment via the 
tool was fair 

50% 

35% 

20% 

5% 

0% 

You would like the tool to replace all standard 
laboratory tests in the future 

5% 

40% 

30% 

20% 

5% 


As it can be seen from the results of the survey, the students reacted very positively and generally enjoyed 
using the proposed tool over their typical manual assessment. As the students were given the exact same time 
to complete the tests of the tool as their common manual tests, most found the tool faster and easier to use 
than the manual test, despite the fact that additional educational material was being given to them during the 
process, and they generally found the assessment process fair and transparent. When asked if they would like 
all of their manual tests to be replaced by the automatic assessment tool, the results were generally favorable 
but more diverse than those of the previous five questions; this indicates that a slower adoption process 
should take place, in order to both resolve any problems prior to completely automating the process and to 
quell the worries of the students. 


3. CONCLUSION 

In this paper, an automated evaluation tool for the laboratory training of engineering students has been 
proposed. The proposed tool has been based on the well-known text comprehension theory of Denhiere & 
Baudet, forming a cognitive profile of the student via a diagnostic test and offering personalized feedback 
and assistance based on the results of this test. This helps to overcome the "one size fits all" approach, which 
is dominant in engineering education today. Similar tools can also be developed based on other 
comprehension theories as well. 

The system proposed in this study has multiple advantages. From an educational point of view, the 
greatest advantage is that the student continues to use the same tool he has been receiving training on, 
without having to result into a classic "pen and paper" type of test or an alien eLearning environment. For 
educators, the advantages of the proposed approach are that it is simple and effective. By making use of 
typical computer files, such as spreadsheets, the modification and or addition of new educational resources is 


302 





International Conference e-Leaming 2014 


a simple and straightforward matter for anyone with basic computer skills. The time that it would require to 
develop such a tool should be less than the time required to manually assess the exercises of a single course. 

Assessment via the proposed tool can be multidimensional, implementing both quantitative and 
qualitative factors. Depending on the cognitive model used, aside from the number of correct, wrong and 
unanswered questions, qualitative factors can also be implemented into the assessment. These can be the 
choice of the additional educational material, if the user skipped the additional material entirely and 
proceeded to the test, the time required to complete each question and the entire test, the number of times that 
the user requested assistance, etc. 

The proposed tool can also be used for online and distance training, provided that the student will be 
instructed to return the file with the results to the educator or informed that the file will be automatically sent 
to the educator once the test has been completed. The only drawback of such an approach is that the 
computer of the user must have a licensed version of the software tool used for the educational process, as the 
system is based on the actual tool itself. 
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ABSTRACT 

This study aimed to explore the attitude and learning effectiveness in game based simulations from college students’ 
perspective. The participants included 189 business college students in Taiwan. The main instiument employed in this 
study was McDonald’s video game. Additionally, participant selection, data collection and analysis; and results relevant 
to the research questions of this study are presented. 


KEYWORDS 

learning effectiveness; digital game, game based learning, attitude toward game based learning 


1. INTRODUCTION 

New generation students are so different from past learners and it seems they cannot live without computers 
technologies. Digital games have a considerable impact on students’ life and learning. Games have been 
defined as an immersive, and enjoyable activity in which a challenging goal is pursued (Freitas & Oliver, 
2006; Kinzie & Joseph, 2008). It is more attractive for students to learn the complex knowledge via digital 
simulation game for college students (Prensky, 2003; Bayliss, 2007;). Generally, games are designed more 
visual, interactive, and focused on problem-solving (Mitchel & Savill-Smith, 2004; Pasin, & Giroux, 2011). 
Games usually include clear objectives, and they provide tasks with multiple difficulty levels to adapt to the 
prior knowledge and skills of learners; therefore, games are considered an effectiveness educational tool 
(Gentile & Gentile, 2008). Moreover, many research have found that game based learning can enhance 
learning motivation and learning outcomes (Dickey, 2011; Sung & Hwang, 2013). 

Thus, this study used college students as participants and aimed to explore what their learning 
effectiveness and attitudes are during the game based learning. 

The following research questions were proposed in this study: 

1. What are the college students’ attitudes toward game -based learning? 

2. What are the college students’ learning performance regarding the subject content knowledge after 
game-based learning? 


2. LITERATURE REVIEW 

Game Based Learning (GBL) is a widespread application teaching or learning strategy in Education, 
especially for teachers to connect learning content and reality that make learners stay in a simulation world. 
In the beginning, the learning potential generated through games has been examined in university classrooms 
since the 1950s, particularly by business schools (Gros, 2007; Lichtenwalter & Baker, 2010). With the 
technology developed, the computer game was developed and applied for educational purposes from 1960s 
(Wolfe & Crookall, 1998). Nowaday, digital GBL courseware has been utilized as an instructional tool in 
cross disciplines ranging, such as international relations, engineering and biology to nursing and social work 
(Lichtenwalter & Baker, 2010). Research also indicated that the use of gems to teach educational content 
raises students’ learning and motivation (Gee, 2005, Prensky, 2003; Sung & Hwang, 2013). 
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3. METHOD 

3.1 Participants and Procedure 

All experimental tasks and data collection were conducted via a computer system. One hundred and eighty - 
nine business major undergraduates (50 males and 139 females) participated in this study. The participants’ 
mean age was 18.9 years. All participants filled out an informed consent form before the experiment. 

Tests and questionnaire were conducted with regard to the learning effectiveness and attitude toward 
game based learning. This study conducted pre-experimental design to process one-group pre-test and post- 
test design. Before the participants played the McDonald’s video game, the pre-test was given in the class. 
The participants were request to playing the played the McDonald’s video game at least half hour per day. 
After a week, the post-test and questionnaire were given to participants in the class. 

3.2 Instrument 

3.2.1 The Framework and the Characteristics of the Simulation Game 

The main instrument employed in this study was the McDonald’s Video Game. In this study, the Mcdonald’s 
was developed by the Italian company La Molleindustria, a free download of Flash games (website: 
http://www.mcvideogame.com/downloads-eng.html). The researchers also translated the English version of 
tutorial guidance in Traditional Chinese version for participants. There are four scenarios in McDonald’s 
game, including natural breeding farm, artificial breeding farm, McDonald's store, and Head office (Fig. 1-4). 



Figure 1. McDonald’s Store 



Figure 2. Artificial Breeding Farm 
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Figure 4. Head Office 


3.2.2 Learning Content Test 

In order to know the learning effectiveness of college students’ content knowledge of business and 
management subject, the researchers gave two set of multiple choices tests, developed by 2 subject experts, 
for participants. Two set of 30-item multiple choice tests were considered the same difficulty and gave 
college students for their pre-test and post-test. The total score of each set test is 30- points. 


3.2.3 Game based Learning Attitude Questionnaire 

One of the goals of this study is to explore the college students’ perception about simulation game based 
learning. To achieve this goal, this study examined how the students perceived the learning experience with 
the simulation games in the course. 

The researchers developed the questionnaire of game based learning attitude to measure the responses of 
participants following completion of the simulation game. This questionnaire comprised a total of 12 
questions. Questions were ranked on a scale of 1 to 5 with 1 representing “strongly disagree” and 5 
representing “strongly agree”. 
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4. RESULTS 

4.1 Result Analysis for Learning Content Test 

After the experiment, the t-test result (t( 188) = 35.827, p<.01) showed the significant difference between the 
pre and post tests (Table 1). The mean score of the pre-test was 6.429 and the mean score of the post-test was 
18.452. Hence, business college students gain better scores regarding the subject content knowledge of 
business and administration after game based learning. 


Table 1 . T Test of learning content tests 



n 

Mean 

Std. 

Std. Error 

t 

df 

Sig. (2- 


Deviation 

Mean 

tailed) 

pre-test 

162 

6.429 

2.643 

.179 

35.827 

161 

.000** 

post-test 

162 

18.452 

4.364 

.296 





**p<.01 


4.2 Result Analysis for the Questionnaire toward Game Based Learning 
Attitude 

The questionnaire was meant to determine how the subjects felt about game based learning. For the 12 
question items, the average score was 4.18 (SD=0.77), as shown in Table 2. Overall, the participants has 
positive attitude toward game based learning. 


Table 2. Mean score of the questionnaire toward game based learning attitude 


Item 

Mean 

SD 

Q1 The Chinese instruction provided makes it easy to use the “game-based simulated 
learning materials.” 

4.29 

.94 

Q2 This “game-based simulated learning materials” is easy to use. 

4.00 

.71 

Q3 Overall, I’m satisfied with the interface design of this “game-based simulated 
learning materials.” 

4.06 

.66 

Q4 This “game-based simulated learning materials” could help me learn the knowledge 
of productivity management in business administration. 

4.13 

.72 

Q5 This “game-based simulated learning materials” could help me learn the knowledge 
of human resource management in business administration. 

3.91 

.87 

Q6 This “game-based simulated learning materials” could help me learn the knowledge 
of marketing management in business administration. 

4.21 

.69 

Q7This “game-based simulated learning materials” could help me apply what I learned. 

4.06 

1.03 

Q8 This “game-based simulated learning materials” could extend my knowledge about 
business administration. 

4.35 

.93 

Q9 Overall, I’m satisfied with the way of learning through this “game-based simulated 
learning materials.” 

4.53 

.72 

Q10 I will be more motivated on learning with game-based simulated learning materials. 

4.30 

.68 

Qll It will be more interesting than lectures only in class if instructors apply game- 
based simulated learning materials on their teaching. 

4.20 

.58 

Q12 I hope that instructors could utilize game-based pedagogy to assist general 
instruction in the future. 

4.10 

.70 

Total mean 

4.18 

.77 


5. CONCLUSION 

Overall, Business college students felt positive toward the McDonald’s video game based learning material. 
Business college students will be more motivated through game based learning from their perspectives in this 
study. Based on the research results, the business students also have better performance on learning content 
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after game-based learning. These results can tell us two things: a) as other studies claimed that GBL can 
promote students’ learning performance (Blunt, 2006; Gros, 2007; Lichtenwalter & Baker, 2010); b) GBL 
can help students link learning content and authentic world. In the future, the researchers might explore how 
student’s learning motivation effects their performance through game based learning. 
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ABSTRACT 

This paper reports on the introduction and use of e-learning technologies to help students study a school subject. The 
research intervention is located within a relatively resourced school in South Africa with students largely coming from 
one racial group. The choice of the school enabled the researchers to focus on investigating the effects of computer-based 
study methods on students skills of studying and their perceived learning experience without the necessity of controlling 
a lot of variables such as socio-economic status, language competence, and so on which have been shown by previous 
research in South Africa as impacting heavily on teaching and learning and student performance. 

This paper describes how technology is used in similar schools as the sample school under investigation, how pedagogy 
has changed practices in the school and how study methods are used. Based on the results of the literature review an 
intervention of introducing free open source software to investigate study practices of school learners was introduced. 
The results of the intervention are reported. 
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1. INTRODUCTION 

Student performance remains one single problem in the South African school system. Up to half the number 
of students who enter primary school do not sit for the final school examination that provides certification. Of 
these students who sit for the national examination only a small proportion qualify to continue their studies in 
higher education institutions. According to the South African National Development Plan, “... we have not 
yet overcome the inequities in educational provision created during the apartheid era. An enormous 
difference in capacity and resources exists in different areas of the country and sectors of society ... we have 
undergone serial curriculum changes at school level, which, while well-intended, have destabilized the pre- 
tertiary education system and placed great strain on teachers. Pressure placed on schools and education 
departments to meet quantitative performance targets is increasingly resulting in teachers spending a large 
proportion of their time drilling learners on how to pass examinations instead of developing learners’ 
intellectual capacity” (CHE, 2014). 

In order to enable school students to achieve other educational outcomes (e.g., critical thinking, problem 
solving and application of knowledge) emphasis should be placed on learning as opposed to mere drilling for 
examinations. Learners need the skills to know how to study a particular field of work and as such exposure 
to a variety of study methods is instructive. This paper describes the intervention made with a group of 
students to improve their study methods through introducing e-learning technologies. First, however, is the 
description of the theoretical framework guiding the intervention. 

The theoretical framework informing the investigation is based on the literature of cognitive science and 
on the literature of the development and application of technology. The theoretical framework thus 
encompasses a focus on cognition and learning, study skills, and e-learning. 
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2. LITERATURE REVIEW 

In order to improve the educational outcomes of students at school, we considered the espoused benefits of e- 
learning in improving the study habits of school students. E-learning is content and instructional methods 
delivered on a computer (whether on CDROM, the Internet, or an intranet), and designed to build knowledge 
and skills related to individual or organizational goals (Clark, 2002). The American Society of Training and 
Development defines E-Leaming as a wide set of applications and processes, such as Web-based learning, 
computer-based learning, virtual classrooms, and digital collaboration. Mallinson provides an expanded 
definition of e-learning as including the use of all digital resources, systems, computers, and electronic 
communication in the support of education (Mallinson, 2013). This study therefore explored the use of a 
software product in changing the study habits of a group of school students. 

The software product used in the investigation had to support how students learn and mediate this 
process. We thus considered learning theory to inform the introduction of e-learning technologies for use by 
school learners for study purposes. Learners need to process the information they receive during the teaching 
encounter and make sense of it. To do this learners make use of cognition. Matlin (2005) describes cognition 
as a mental activity which includes the acquisition, storage, transformation and use of knowledge. Cognition 
can also be described as the mental processes of knowing which will include aspects such as awareness, 
perception, learning, memory, thinking, reasoning and judgment (Train, et al., 2007). These cognitive 
processes are at play when a learner studies a school subject. The pedagogic strategy we employed was to 
help learners think about the cognitive processes and strategies they use when studying. Employing such 
metacognitive strategies to the learning process has been found to enhance learning (Foster, Sawicki, 
Schaeffer and Zelinski, 2002; Matlin, 2005; Jaeger, 2007). Foster et. al. (2002) distinguishes between 
metacognitive and non-metacognitive learners. The main difference between these two types of learners is 
that metacognitive learners are aware of how they learn best and that they evaluate and regulate each learning 
experience. Non-metacognitive learners use monotonous learning skills. 

Metacognitive learners use a variety of activities (e.g., cognitive maps) to evaluate their learning. The 
pedagogical strategy used in this investigation was to teach learners about using cognitive maps (mind maps) 
as a study method and to use e-learning technologies to support this process. According to Merritt (2008) 
teaching study skills to a learner is an essential part of the education process. Effective study skills are an 
outstanding characteristic of all high achieving students. Using effective study skills enable learners to adapt 
to various teaching methods and instructional approaches and thus enhance student performance. 

Learners develop their own study strategies during schooling. Teachers need however to support learners 
and help them to learn independently. Exposure to various methods of studying therefore provides a learner 
with a repertoire of methods to use in alignment with their study preferences. The group of students used in 
the investigation were provided with the variants of the SQ3R study methods but specifically taught to use 
mind mapping. Mind mapping facilitates active and collaborative learning, enables the learner to make use of 
their full range of cortical skills, and facilitates the conceptualization of richer and broader associations that 
enhance learning (Budd, 2004; Smith, 2008). 


3. RESEARCH METHODOLOGY 

The participants in this research study were learners in Grade 11 in the secondary school where the first 
author teach. All the learners that are participating have Business Studies as one of their choice subjects. A 
total of 93 Grade 11 learners (52 females and 41 males) took part in the research. Learners who took part in 
the investigation are predominantly from one racial group (table 1). Consent from both the principal of the 
school and the parents of the learners were sought before requesting learners to participate in the 
investigation. 
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Table 1. Composition of participants in the investigation. 


Grade 

White 

Black 

Coloured 

Total 

11 

239 

1 

4 

244 


All learners in table 1 were taught how to use study methods and mind mapping whilst they were in grade 
10. This cohort of students was randomly divided into an experimental and control groups. The experimental 
group was taught after school hours to use a free mind map software package (FreeMind) for a duration of ??. 
To espouse collaborative learning and metacognition, learners were provided with tasks of working in groups 
and to reflect about the methods and tools they use for studying. 

The research design involved administering a pre- and post-test instrument on both experimental and 
control groups. The post-test was administered after introducing the use of e-learning technologies on the 
experimental group. Both pre- and post-tests made use of the Learning and Study Strategies Inventory - High 
School Version (LASSI-HS) (Weinstein & Palmer, 1990). The LASSI-HS is a 76-item self-report instrument 
constituted by 10 subscales (see table 2). The LASSI-HS has appropriate validity and reliability for use to 
determine the study methods of school learners (Tinsley, 2000; Everson, Weinstein & Laitusis, 2000). The 
data of the pre- and post-tests were analysed to provide an indication of whether the use of e-learning 
technologies had an impact on the study skills of the learners or not. 


Table 2. Subscales of the LASSI-HS scale 


Nr. 

Subscale 

Description 

1 

Attitude 

Attitude measures the learners’ attitude and interest in education and school. 

2 

Motivation 

Students’ diligence, self-discipline and willingness to work hard is measured 
with this subscale 

3 

Time Management 

This scale examines the students’ use of time and management principles. 

4 

Anxiety 

Anxiety address the degree to which students worry about school and their 
performances. 

5 

Concentration 

Concentration is the ability of a student to pay close attention to academic 
tasks. 

6 

Information Processing 

This scale measures several aspects such as the use of mental imagery, verbal 
elaboration, comprehension, monitoring and reasoning. 

7 

Selecting main Ideas 

The ability of the student to pick out important information for further study. 

8 

Study Aids 

This is the scale that examines the degree to which students create or use 
support techniques or materials to help them learn and remember new 
information. 

9 

Self-testing 

Self testing is the scale that measures the students’ ability to review and prepare 
for classes. 

10 

Test-strategies 

This scale focuses on students’ approaches to preparing for and taking quizzes 
and tests. 
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4. RESULTS OF STUDY 

Tables 3 and 4 provides the results of the pre- and post-tests for the experimental and control groups. 


Table 3. post-test LASSI-HS: experimental group. 



ATT 

MOT 

TMT 

ANX 

CON 

INP 

SMI 

STA 

SFT 

TST 

| Before the intervention | 

Mean 

31.0 

27.6 

20.6 

22.4 

24.6 

26.4 

16.2 

23.7 

23.6 

25.1 

Mode 

32 

34 

24 

25 

29 

27 

15 

22 

22 

29 

Median 

32 

27 

21 

23 

26 

27 

15 

24 

22 

26 












| After the intervention | 

Mean 

31.4 

28.0 

23.3 

20.1 

25.5 

28.2 

18.9 

25.9 

23.8 

25.2 

Mode 

28 

34 

22 

20 

29 

29 

18 

28 

22 

29 

Median 

32 

27 

25 

20 

26 

28 

18 

26 

22 

26 


Table 4. post-test LASSI-HS: control group. 



ATT 

MOT 

TMT 

ANX 

CON 

INP 

SMI 

STA 

SFT 

TST 

| Before the intervention j 

Mean 

30.4 

27.1 

21.4 

23.6 

23.4 

27.8 

16.7 

24.1 

24.6 

24.4 

Mode 

35 

18 

16 

18 

27 

30 

17 

24 

24 

20 

Median 

32 

27.5 

21 

22.5 

25 

28.5 

17 

24 

24 

25 












| After the intervention j 

Mean 

28.6 

26.7 

20.9 

23.4 

23.1 

27.4 

16.7 

23.6 

24.1 

24.1 

Mode 

35 

18 

16 

30 

33 

22 

17 

24 

23 

30 

Median 

30 

26.5 

21 

22.5 

25 

28 

17 

24 

24 

25 


Both these tables provide a better indication of the differences in the scores of the individual scales before 
and after the intervention. The central measures of tendency can be used to determine certain aspects 
regarding the experimental group. Before the intervention took place, the learners in the experimental group 
had a mean score of 26.4 regarding the information procession scale. After the intervention the mean score 
for the same scale was 28.24. The selection of main ideas scale of the LASSI-HS also increased from 16.28 
to 18.88. The score on the use of study aids, such as computers, also increased from 23.72 to 25.88. Using 
the data from the control group, the mean from the information processing scale decreased from 27.8 to 27.4. 
The scale for the selection of main ideas indicated that the score also decreased from 17 to 16.4. Finally the 
scores for the use of study aids scale decreased from 24 to 23.6. Based on these results, it shows that there 
are slight increases regarding the central tendency measurements for the experimental group. The central 
tendency measurements regarding the control did not show any increases between the pre- and post-tests. 


5. CONCLUSION 

Analysis of the results of the control group indicates no increases in measures of the ten subscales of the 
LASSI-HS. On the other hand, all the measures within the experimental group, except anxiety, show slight 
increases. These increases could be attributed to the intervention introduced in this investigation. 

The use of structured approaches to inculcate improved study habits amongst school learners is therefore 
supported by the results of this study. 
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ABSTRACT 

The paper is aimed to evaluate the possibility of applying new methods and tools of more effective educational 
processes, with an emphasis on increasing their quality especially aimed on educational processes at secondary 
schools and universities. There are some contributions from practice for the effective implementation of time 
management, such as reducing unnecessary delays, in order to increase the efficiency of education from the 
perspective of the provider of educational services - secondary schools and universities, as well as from the state 
as evaluating authority (accessibility of educational services, costs savings, time, etc.). The Small business check 
- up methodology can be the right and cheaper way for completing or replacement of actual monitoring system 
provided once per year in paper form and only in selected groups. The basic idea of this approach is to establish 
an appropriate system for exchange of information between individual schools and also among the state schools 
and institutions. The paper also deals with differentiation between quality level of schools across individual 
regions in Slovakia. 


KEYWORDS 

Educational process, e-learning, small business check - up, evaluation 


1. INTRODUCTION 

The importance of information and communication technologies is associated with various aspects of 
life of modern society. Their application can be found in almost every segment of the work. The 
secondary education is not an exceptional segment, in which there is increasingly growing importance 
of e-learning support for education. There is an increasing demand for improvement of educational 
process at secondary schools and universities, as well as the ever increasing demands of external 
companies for graduates who master the latest technology require continuous innovation in the 
educational sphere and they need to be sure that every school with similar branch of study has the 
same level in quality of education. 

From this reason the former approaches are not appropriate anymore because losing any chance 
for flexibility moreover the evaluation period is too long what causes delays in counter measure 
actions which have to be implemented as soon as possible. The most possible and reachable changes 
in these systems can be founded in base of some methodologies running at information technologies 
such as the method for development support in small companies “Small business check - up”. 


2. SMALL BUSINESS CHECK - UP TOOL METHODOLOGY 

Diagnostic tool for enterprise "Small business check-up" was developed by the Council for leadership 
and management development in British Columbia, in abbreviation LMDCBC ("The Leadership and 
Management Development Council of British Columbia"). LMDCBC saw in the tool "Small business 
check - up" (further SBCH) a crucial initiative to improve leadership and management capacity of 
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small and medium-sized enterprises because the tool has increased awareness of the possibilities of 
innovation in society and the present investment costs that are spent on these innovations. 

SBCH by its structure belongs to the so-called Business diagnostic tools (questionnaires or 
checklists) that provide a framework to help business managers and consultants in reviewing the 
activities of the company and identify potential areas of improvement. There were developed plenty of 
diagnostic tools over the years. These tools are mostly aimed at larger companies and associated with 
the sale of specific services such as consulting services or products, such as accounting programmes. 

Therefore, the main aim in developing SBCH was to go beyond that what is currently available 
and create a new diagnostic tool with the following particulars: 

to be available on Internet with no additional cost for the user, 
to be aimed primarily at small and medium-sized enterprises, 

to be action-oriented, and not only to determine the current state of society, but also to 
establish a clear plan as currently identified condition innovate to improve the functioning of 
the enterprise. 

As it was mentioned above the SBCH tool is fully available on free version through a web 
browser. Under the program the user is advised to answer firstly on many questions about the focus, 
background, objectives, strengths and weaknesses and management practices in the company, as well 
as the skills of the managers and the employees. Subsequently the tool allows user to compare his 
answers with other companies (of course in an anonymous way). It also provides a framework for 
reviewing company operations and identify opportunities for improvements. Once the user selects a 
number of priorities for improvement, SBCH will help to prepare a strategy and action plan for their 
implementation. Illustration of instrument SBCH web environment is shown in the following Figure 1. 



Figure 1. Web environment of „Small business check-up" tool (Source: www.smallbusinesscheckup.com) 


The instrument was designed to serve as a guide for improving management capabilities and 
efficiency of operational processes. Therefore it works only as a supplement in the strategy 
management and it should not be regarded as a substitute for commercial diagnostic tools such as 
accounting programs mentioned. 


3. CURRENT METHODOLOGY FOR COMPARISON OF SCHOOLS 

Regarding the evaluation of secondary schools in Slovakia it is currently carried out in the form of 
testing the knowledge level of students in the schools concerning basic subjects such as: Slovak 
language and literature, mathematics and foreign languages. Based on the results from individual 
subjects there were subsequently evaluated the average success rates for all schools and then in a 
county. In previous years, testing has shown that there are considerable differences in the educational 
level of individual students of secondary schools, as well as in 8 self-governing regions in Slovakia. 

There are more than 720 secondary schools present in Slovakia with average grades ranged from 
8.1 to 6, the disparities occurred in the level of secondary education according to various region. In the 
following figure 2 (MAT-mathematic, SL-English language) we can see that the biggest difference in 
the level of knowledge is in mathematics. This fact is a negative indicator from the point of view of 
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further educational process mainly because of differences in knowledge level and preparedness of 
secondary school students in continuing their studies at university. 
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Figure 2. Average success rate in knowledge of Slovak language and mathematic on secondary schools by 
region(Source:http://www.nucem.sk/documents//25/maturita_2013/vysledky_analyzy/Pr%C3%ADloha_k_Spr% 
C3%Alve_o_priebehu_a_v%C3%BDsledkoch_EC_a_PFIC_MS_201 3_final.pdf) 

Higher difference in the level of knowledge occurred when we compared schools according to 
their study program (figure 3) where we can see deeper knowledge gap between the level of 
mathematics knowledge as well as Slovak language and provider (figure 4), where we can see 
between students of different types of schools. This trend is caused to some extent due to the quality 
of educational staff (teachers and teaching aids), which the types of schools have in diposal. 
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Figure 3. Average success rate in knowledge of Slovak language and mathematic on secondary schools by 
type(Source:http://www.nucem.sk/documents//25/maturita_2013/vysledky_analyzy/Pr%C3%ADloha_k_Spr%C3 
%Alve_o_priebehu_a_v%C3%BDsledkoch_EC_a_PFIC_MS_201 3_final.pdf) 
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Figure 4. Average success rate in knowledge of Slovak language and mathematic on secondary schools by 
provider (Source:http://www.nucem.sk/documents//25/maturita_2013/vysledky_analyzy/Pr%C3%ADloha_k_ 
Spr%C3%Alve_o_priebehu_a_v%C3%BDsledkoch_EC_a_PFIC_MS_201 3_final.pdf) 

The basic principle of assessment does not change over the years - publicly available quantitative 
data are divided into five groups: 1) education (number of students, teachers, quality of teaching 
staff); 2 attractiveness of study (interest in the study, the proportion of foreign students, mobility, 
unemployment); 3) research (numbers of publications and citations, average citations); 4) PhD 
(proportion of graduates share of the student population, the scientific performance ratio of the 
number of PhD students); and 5) the grant percentage (domestic grants, international grants, the total 
income from grants for creative worker). 

As we can see according to these criteria, many similar study branches at universities it do not 
reflect the level of knowledge that students should achieve in these fields of study. Evaluate mainly 
the number of students, teachers and the level of scientific activity or ability of graduates to get job 
after study. Therefore, for this reason it would be appropriate at least on some types of schools to 
establish informative comparison of the level of students' knowledge for example in the same fields as 
economics, management, mathematics, etc. 


4. HOW CAN SMALL BUSINESS CHECK- UP SUPPORT 
EVALUATING SYSTEM 

As it was mentioned above in major systems and methodologies for comparing in secondary schools 
and universities are not very time and cost-saving. They can be carried out only once a year and 
evaluation criteria are mainly in universities focused on performance and not on knowledge and skills 
of students and teachers. Implementation should be carried out as a main directive from ministry and 
basic tool built on SBCH methodology should be available on-line by web page based on web form 
application. 

Secondary schools - with using the SBCH methodology thus developing an online form with the 
output to. NET graphical presentation of the results could be easily and quickly controlled the current 
level of preparedness of students at individual secondary schools. It would also be possible to carry 
out testing several times a year for example at the beginning of the individual grades, the bi-annual 
evaluation and year-end reviews. This form of continuous assessment could provide schools and 
government institutions timely with information about the progress of students in the tested subjects. 
Secondary schools would also receive a tool for continuous evaluation of their performance compared 
to other schools in the state and thus would have the opportunity to take corrective action even during 
the course before final assessment. 

University - the modified SBCH instrument would be more effective as an additional evaluation 
tool in universities. On the one hand it could be used as a classic online tool to assess the knowledge 
level of students in similar branches of study and related theory. By these means the theoretical base 
could be effectively evaluated for individual universities. On the other hand the instrument could be 
used for testing the practical knowledge level of students in solving practical examples from business 
practice through case studies (in form of test questions). Also such testing would be possible to 
evaluate not only students but also teachers of universities and their ability to provide students with 
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relevant information necessary for practice. With this could be better described a point from 
universities ranking regarding graduate unemployment (why they are unemployed). Universities could 
reach by such testing a continuous image of balance between theoretical knowledge and practical 
training in the followed subjects. For university to be on the bottom of the list the annual evaluation 
could be test resource of the data in areas in which they could improve continuously during the year. 

Pedagogical perspective - SBCH methodology could help predominantly to teaching staff in 
schools. Because its original purpose was to measure, benchmark and improve the skills of strategic 
management, this procedure can be applied also to improving the skills of the teaching staff. Staff of 
universities don’t need to have examinations of pedagogical minimum and thus their teaching 
methodology resulting from their own approach and personality. By succession of well-chosen 
comparative test questions they could find potential week points in their teaching practices and 
thereby improve their skills and could better interpret knowledge to students. 


5. CONCLUSION 

The effort of many developing countries mainly in the European Union is focused on the development 
of the school systems. The number one priority is to prepare graduates for their immediate application 
in practice after graduation. Graduates taking up practice often encounter problems due to lack of 
preparedness for the position in the company even if their field of study was aimed the position they 
were applying for. Due to varying degrees of preparation from different schools with the same fields 
many requests from candidates are filtered according company's experience with concrete school. It 
can thus be said that an equivalent degree in the same field does not necessarily guarantee equal 
chances in the labour market. Of course it is a relatively wide area where this issue is concerned but in 
its successive solving can also help testing methodology and continuous deeper comparison of results 
and teaching methods which we have presented in the paper. The SBCH methodology has been 
successfully implemented as a cheap and fast support for improving the skills of strategic management 
in smaller companies. If we take these small businesses as individual schools where appropriate 
departments and subjects taught to them and teachers as responsible managers we reach clear picture 
of options that this methodology could contribute to improve the level of secondary schools and 
universities not only in developing but also developed countries. 

This contribution was undertaken, as parts of the research projects VEGA 1/1321/12, VEGA 
1/0895/13, VEGA 1/0421/12, KEGA 052ZU-4/2012. 
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ABSTRACT 

In this paper we will investigate the way that the technological progress and the Informatics contributed greatly to the 
field of Archaeology. There will be analyzed the terms of virtual archaeology and virtual reality in archaeology and there 
will be an extended reference to the applications and the computer graphics that archaeologists could use for their own 
scientific purposes. It will be attempted to be shown the way that computer graphics can create not only an authentic copy 
of an archaeological find but can function also as a useful tool of learning for new archaeologists and the broader public 
that is interested in the ancient past. 
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1. INTRODUCTION 

In the field of culture the new technologies offered new potentials of management and presentation of the 
cultural heritage and redefined the role of the cultural organisms and institutions that deal with the 
documentation, the preservation and the promotion of the cultural asset. The archaeologists used the new 
technology, the computer graphics, in order to visualize the archaeological data in a way, easy to understand, 
not only for themselves but also for the public that was interested in the archaeological finds. The “virtual 
reality in archaeology” or the “virtual archaeology” as it was named, could open new ways in the scientific 
community concerning the research and the presentation of the ancient remains. It created various 
applications of simulation, depiction and representation not only of the excavated finds but also of the modus 
vivendi of the ancient civilizations. The images that were created for the reproduction of the ancient past or 
the monuments, constituted a subject of skepticism concerning the authenticity of the archaeological 
information included in these images. The effort to represent the ancient past with the new technology poses 
new, controversial problems over the documentation and the credibility, arising new issues and speculations 
(Molyneux, 1997:1). This paper presents the created virtual environments based in archaeological data that 
can constitute a new, contemporary and potential learning environment developed for future professionals in 
archaeology. The new virtual environment can be used also for learners of all ages as a new method of 
informal education in cultural organizations and spaces such as in the museum. 


2. THE 3D REPRESENTATION 

The 3D graphics consist one of the major achievements in the field of Informatics. In order to be created the 
contents of a “scene” or for the synthesis of a three- dimensional representation, there are followed methods 
of calculations that derive from the exact sciences like mathematics and geometry for the rendering of the 
texture (texturing), the illumination of a 3D scene or in the case of motion (animation). From their presence 
and then, the 3D graphics have seen a rapid development, even in an artwork’s display (Polymeropoulou, 
2010). With the new technology, many sciences wanted to use the 3D graphics in order to visualize objects, 
ideas or situations that so far it was impossible to be represented. This technical and hypothetical element that 
forms the imitation of the real created the meaning of “virtual”. 
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2.1 What is Virtual or “Qu’est que le virtuel”? 

The new trend of visualizing elements, data or ideas had a broad effect answering not only recreational 
purposes but as well as scientific and educational. The exact meaning of the word “virtual/ virtuel” is defined 
as the potential, something that can exist but at the same time is not real. The word derives from the Latin 
virtuelis>virtus which means power, force. The philosopher Gilles Deleuze uses the word “virtual” so to 
describe something that every object carries with it that is not its reality, nor only that it could be but rather 
what it is supposed to be. The “virtual” is used to indicate a possible situation that it could be substantial 
(Deleuze, 1968). According to the French philosopher, the virtual reality is understood as the possible reality 
with two reverse meanings of the real. Since, from one hand the VR consists of a simulation of the natural 
environment, on the other hand whoever simulation from her nature simulates a real object or fact, the 
experience of a possible reality that depends on the visual effects of the computer, creates a environment with 
double character. And here is the oxymoron: The environment of virtual reality is real meaning it exists and 
is operative, but on the other hand is not real since it is a simulation of the reality (Levy, 1999:26). Due to the 
fact that the word “virtual reality” and “virtual” puzzled many philosophers and researchers, many prefer 
other terms for the rendering of the word. So they rather talk about “compound experience” (Sanchez et al., 
2001; Beroggi et al., 1995; Loomis, 1993) or “compound environment” (Durlach & Mavor,1995; Jayaramb et 
al.,1997), “simulation technology” (Psotka,1995), “artificial reality” (Biocca & Levy,1995) or just 
“cyberspace” (Hayles,2001). Nevertheless, the word “virtual reality” has been prevailed almost generally. 

2.1.1 The Virtual Archaeological Environment 

The introduction of the computer in the documentation, the depiction and the presentation of the 
archaeological data changed the archaeologists’ way of thinking, arising new urgent questions about the 
methodology of the research, the knowledge and the spread of the culture. The technological achievements, 
depending on the needs and the archaeological theories that come up in every time period, come to cover the 
needs of the archaeological and excavation survey and methodology. We observe the gradual transition from 
the simple observation through comparison and the data analysis to the visualization and the reconstruction 
or/and the simulation of the ancient pasts, by using the continuously evolving programming languages and 
the artificial intelligence. The 3D graphics were used especially for the simulation of a virtual archaeological 
dig. Since Archeology is a scientific field that is based on excavation and the field survey, the majority of the 
applications are focused on simulating the archaeological dig with realistic problems that an archaeologist 
may have to deal with (Fig. 1). A virtual dig consist of an excellent instructive tool for the new, future 
archaeologists since the excavation is a destructive method (Polymeropoulou, 2014:45) without allowing for 
a margin of erroneous decisions and movements. The new archaeologist is placed virtually in the 
environment of the excavation and deals with challenges such as how to interpret the elements of the ancient 
human behavior that the soil reveals during the dig (Slator and Associates, 2006:71). For the best possible 
simulation of the virtual dig, the user - excavator should be in a virtual excavation field where the data are 
based on real excavation scenarios. Moreover, the faithful depiction of a dig offers multiple searching 
potentials to the archaeologist himself for the detailed analysis of the archaeological particulars. 




Figurel. Simulation of an archaeological dig (Dunn, 2002:122 - 23) 

The technology VR permits the archaeologists to use a relevantly economical equipment so to accelerate 
the excavations and to preserve more analytical, accurate and accessible geometrical data of the 
archaeological finds and the location where there were found (Leymarie et al., 2000:3). In a virtual 
environment all the elements are comparative, dynamic and interrelated (Forte & Pescarin, 2006:4). The 
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archaeologist in a virtual learning environment is able to have access in virtual copies (models) that bear the 
same information as the authentic and can study/ process data with no restraints or without the fear of 
destructing an ancient object. Even more, he is given the opportunity with the virtual environment to return 
in it long time after and to re-examine his data, making new interpretations and assessments in order to 
strengthen or to refute his theories. As a result, the creation of realistic 3D models synthesizing a virtual 
environment, according to the data recorded in situ during the dig, is of major educational importance. In this 
virtual environment, the new archaeologist learns how to excavate, to study, to interpret and finally to 
understand the ancient past by optimizing his methods and techniques in the excavation field. Besides, the 
real dig is a non reversible procedure and every datum is of crucial importance so to fulfill the image of the 
distant past. 



Figure 2. The case of virtual representation of the monument according to real archaeological data. The Laconia 
Acropolis Virtual Archaeology project (LAVA) (Getchell et al., 2006:6) 

The archaeologists used the 3D graphics so to represent the ancient monuments (Fig. 2). More 
specifically, they attempted, through the control of different hypothesis and conjectures, to experiment with 
the texture, the illumination, the location of observation or the form of the model (Godin, 2002). During the 
representation of the ancient monument the 3D graphics can depict the different phases of life of the 
monument and the procedure of its discovery. The scientists have been aided substantially in the field of 
conservation and restoration of finds, movable and non movable (Velios & Cummings, 2001:10) (Fig. 3). 



Figure 3. The case of the digital conservation of the so called statue of “harassed Hercules”. Modeling of the down half 
that stands in Attaleia Museum in Turkey and the completed up half in the Fine Arts Museum in Boston, USA.(Gruen, 

2009:302) 

During the digital conservation, the conservator has no longer direct contact with the object and the 
conservation can be done virtually. In this environment, the conservator can be educated since there are 
proposed different solutions for the conservation of a broken vase or the restoration of a monument and can 
calculate all the possible versions (Forte& Pescarin, 2006:4). The digital completion and the aesthetical 
restoration have a low cost. That means that the heavy, fragile and immovable objects can be easily and 
equally conserved as the small and light objects. In the field of visualization, the 3D graphics convert the 
created or selected data to visual representations. In a virtual environment the user can experience a 
completed integration of data and information. The new technology put on a small revolution on the way that 
the information could nowadays be stored, retrieved and presented. 
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Stanford Digital Forma Urbis Romae Project 




Figure 4. The internet data base of archaeological content. The example comes from the project “Stanford Digital Forma 
Urbis Romae Project” under the auspices of Sovraintendenza ai Beni Culturali del Comune of Rome. (Roller, 2008: 158) 

The systematic storage of natural and chemical characteristics, descriptions of typology, historical 
information and cultural data for the objects of the cultural heritage leaded to the creation of cultural data 
bases (Fig. 4). These cultural data bases contain information about the cultural assets, monuments and 
museums, that are open to the public via internet. The power and the multiple potentials of the internet in 
collaboration with the 3D technology resulted to the creation of virtual museums. The virtual museum in the 
website or the virtual environment in the museum, addressing to the museum visitors, differs towards the 
structure and the purpose of existence, from a system of virtual reality that is used by the scientists. 


3. CONCLUSION 

The applications of virtual archaeology are based on the narration of a simple, realistic and interactive 
system. The participant - visitor, immerses and gain an enjoyable experience while he learns through his 
participation in the application. On the contrary, the scientists rely on the complicated Data Bases that are 
constantly renewable and are used by the community. The aim of the scientists is to study and to research 
their data, following trustworthy methods and techniques. The purpose of the virtual environments is not to 
imitate the reality but rather to contribute in order to understand the reality (Barcelo, 2001:231). As a result, 
the issue is not in what way virtual reality can become reality but rather how the virtual can enhance in a 
novice and alternative way the experience of reality (Gillings, 2002). Even though many objected and 
discredited to the meaning of the term “virtual archaeology”, the need of a scientific integrity and credibility 
of the applications of virtual reality whether for a broader public or the scientific community, raised major 
issues of documentation and standardization. Towards this, the London Charter in 2006 enacted the purposes 
and the principles of using the methods of 3D visualization in regard to integrity, reliability, transparency, 
documentation, standards, sustainability and access. It becomes clear that in both professional, scientific 
applications and in those for the museum visitors, the virtual environment can be seen as a means that will 
help to bring home the knowledge. A part of this knowledge can be achieved if there will be presented the 
sources, the methods, archaeological and technical, that aided to the visualization (Ryan, 2001:245). 
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ABSTRACT 

Replacing lecture-based learning content with online information can augment learner-content interaction and facilitate 
greater mastery over a subject. The success of online delivery will depend on the readiness of learners to use and accept 
technology as well as the readiness of the organizational infrastructure to support a learner-centric learning culture. 
Successful implementation and adoption of online learning can be indicated by technology acceptance. In this project, 
lecture-based learning content for a healthcare vocational training course in West Africa was replaced with e-learning and 
applied to learners who had little or no prior computer experience. E-learning content was combined with practical and 
peer-based pedagogy to provide a blended learning course. This paper describes the acceptance of technology-assisted 
learning during the project. 
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1. INTRODUCTION 

E-learning can provide flexibility and cost benefits over traditional teaching methods (Bouhnik 2006) by 
increasing the quality of learner-content interaction (Anderson 2003). A buffet of online and offline 
pedagogies is increasingly expected from vocational learners as learner-centric quality control becomes more 
important (Hood 2013). Employing learner-centric blended vocational training allows strategies to increase 
learner control and engagement (Johnson et al. 2010), circumvent learning degradation through cost-effective 
refresher training (Chizmar 1999) and reduce attrition (Salmon 2004). 

The prevailing paradigm for fieldworker development in The Gambia before this project was classroom- 
based, lecture-driven teaching incorporating information delivery and practical skill-based pedagogy. 
Refresher training was rarely possible due to time and cost constraints. This paper details the application of 
blended learning to vocational training in the context of Gambian medical fieldwork, changes in IT 
confidence in new IT users and subsequent technology acceptance. During this project the course content was 
unchanged, but the format of delivery was altered from classroom-based delivery to online delivery, with 
practical pedagogies unchanged. During implementation of the course, fieldworkers were asked about their 
perception of the course to determine technology acceptance. 


2. TECHNOLOGY ACCEPTANCE 

The Technology Acceptance Model (TAM) (Davis 1989) with subsequent adaptations (Venkatesh et al. 
2003, Liaw et al. 2006, Venkatesh 2010) is the most widely cited and validated model of determinants to 
acceptance and subsequent successful adoption of technology. TAM adapts the Theory of Reasoned Action 
(TRA) which indicated that intention precedes behaviour (Ajzen 1980). TAM can be applied to technology- 
assisted learning: perceived ease of use and perceived usefulness (Lee 2005, Liaw 2008) of e-learning 
indicate intention to reuse e-learning, and therefore acceptance. Learner self-efficacy affects perceived ease 
of use and engagement (Ong 2004). Perceived enjoyment also serves as a motivator, incorporating flow, 
playfulness and enjoyment which affect intention to reuse e-learning (Byoung-Chen et al. 2009, Padilla- 
Melendez et al. 2013). 
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2.1 Method 

The existing core vocational syllabus for fteldworkers was modified to replace lecture -based content delivery 
with e-learning modules and online examination. Interactive diagrams, audio, video and text were used 
online to enhance learner-content interaction within each e-learning module. Problem-based offline 
assignments for self-regulated study accompanied each e-learning module. Peer interaction was facilitated 
through the addition of scheduled group discussion workshops using real case study examples for each 
module. Practical training and marking procedures remained unchanged in the new syllabus. The UK control 
group consumed e-learning modules only. 

Participants were 195 medical fteldworkers native to The Gambia who participated in both online and 
offline components of their vocational training. A paper-copy questionnaire was provided to all fieldworkers 
before (with 21 questions, n=110) and after (with 31 questions, n=140) learning was undertaken, including 
demographic information and questions on satisfaction and technical functionality which are not included in 
this paper. 74% of respondents were male, and 71% of the group had no experience with online training. A 
comparable front-line operational group in the UK was given the same questionnaires before and after their 
vocational training, with 10 questionnaires returned. This control group was 57% male and 74% had 
experienced online training before. 

Participants responded to items on perceived usefulness and perceived ease of use on a 5 -point Likert 
scale ranging from 1 - strongly disagree to 5 - strongly agree. All questions were in English and were 
adapted from published sources with advice from the fieldworker manager. Pearson correlation was 
completed to determine relationship strength and direction. 

2.2 Results & Discussion 

Preference for online training was much higher in the Gambian fieldworker cohort than the UK front-line 
group, and fieldworker preference for online training increased through the use of online training (Figure 1). 
Reported preference remained low (below the response midpoint of 3) in the UK group, and remained 
positive in the Gambian group. All mean responses from Gambian Fieldworkers were over the midpoint of 3, 
indicating positive satisfaction. 



Figure 1. Preference for online training 

Reported confidence increased in the fieldworker group through the consumption of e-learning, whereas 
the high level of confidence reported by UK workers remained unchanged through the project (Figure 2). 
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Gambian Fieldworkers 


Figure 2. Computer self-efficacy changes through the use of technology-enhanced learning 


Table 1. Reported technology acceptance determinants 


Question 

number 

Mean 

Pre- 

learning 

Score 

Mean 

Post- 

learning 

Score 

Question wording 

Correlation (Post-course answers only) 






3.2 4.1 2.3 

6.1 

7.5 

7.4 

7.2 

6.7 

7.6 

3.2 

3.59 

3.88 

I prefer online training to classroom 

0.4 0.3 

0.3 

0.4 

0.2 

0.1 

0.2 

0.1 




training 








4.1 

3.96 

4.13 

I feel confident using computers or 

0.3 

0.1 

0.3 

0.1 

0.0 

0.4 

0.2 




mobile devices 








2.3 

❖ 

4.10 

The method of delivery was easy to 

- 

0.4 

0.4 

0.5 

0.4 

0.4 

0.5 




use 








6.1 

* 

4.07 

The module was easy to access 


- 

0.5 

0.5 

0.4 

0.4 

0.5 

7.5 

* 

3.99 

The module was easy 



- 

0.5 

0.4 

0.4 

0.5 

7.4 

❖ 

4.47 

The module was enjoyable 




- 

0.7 

0.5 

0.7 

7.2 

* 

4.47 

The learning was relevant 





- 

0.5 

0.6 

6.7 

* 

4.02 

The content** was useful 






- 

0.5 

7.6 

* 

4.40 

I intend to reuse the module for 







- 




information purposes 









* Question was only asked in the post-course questionnaire 
** Content was specified as glossary and reference materials 


E-learning enjoyment correlated moderately with ease of use and ease of access, but not with preference 
for online delivery or with self-efficacy (Table 1.). All responses for perceived usefulness and perceived ease 
of use were above the midpoint, indicating technology acceptance, however, the strongest correlations was 
between relevance and enjoyment. Intention to reuse online learning correlated strongly with perceived ease 
of use and perceived usefulness responses but not with preference or self-efficacy, supporting the technology 
acceptance model. 

Self-efficacy correlated strongly with computer usage outside work which correlated strongly with use at 
work (Table 2.), indicating that fieldworkers confident with computers use them as part of their daily lives, 
either inside or outside work; those who are less confident are less likely to use computers. Self-efficacy had 
little effect in this study, as participation was mandatory (as part of vocational training) and fieldworkers 
reported enthusiasm for the blended learning course, therefore observations based on participation or 
engagement were not possible. 
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Table 2. Reported computer self-efficacy 


Question 

number 

Mean 

Post- 

learning 

Score 

Question wording 

Correlation (Post- 
course answers 
only) 




4.1 4.2 4.3 

4.1 

3.96 

I feel confident using computers or 
mobile devices 

0.7 0.4 

4.2 

3.68 

I use a computer outside work 

0.6 

4.3 

3.43 

I use a computer at work 

- 


3. CONCLUSION 

Introducing technology-enhanced learning as part of mandatory vocational training for Gambian fteldworkers 
provided a blended -learning outcome, which was accepted by fieldworkers. Fieldworkers reported preference 
and self-efficacy increases during the project, and reported technology acceptance as a positive intention to 
reuse e-learning correlated to perceived ease of use, perceived usefulness and perceived enjoyment, 
supporting Davis’ technology acceptance model. Further research is needed to determine the variables that 
influence perceived ease of use, enjoyment and usefulness within the fieldworker community, the social 
influences on technology acceptance in West Africa and the effect of infrastructure on perceived ease of use. 
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ABSTRACT 

This paper describes an existing web-based learning platform named “Third Place of Learning” (TPL) - “Dritter Lernort”. 
This project’s aim is to connect the system of vocational education with digital media by a web-based learning platform. 
TPL supports student’s digital learning by means of interactive examples and exercises. Learning material is produced 
along a didactical concept. The technical structure is as well developed along a didactical concept as the learning content. 
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1. THE THIRD PLACE OF LEARNING 

Germanys current shortage of skilled personnel evolves to a barrier, which negatively affects the national 
economic potential. The project “Third place of learning” (TPL) [TPL14] aims to qualify more trainees in 
stopping this negative trend and tries to optimize the quality of vocational education. Exploring the potentials 
of digital media for vocational education is a further aim of this project. The collaboration partners develop a 
concept for a web-based platform, which provides training content and e-learning tools for technical product 
designer. These e-learning tools can be used by vocational schools, business companies and trainees 
[HeLM14.2], Using digital media to provide teaching and learning material with an e-learning platform for 
trainees and business-instructors has a huge potential. It is fact that more than 70% of young people between 
10 and 18 years use the internet to search for job and trainee information [HeLM14.2]. 

1.1 Digital Media and Dual Vocational Education 

The dual vocational education system exists in three European countries: Germany, Switzerland and Austria. 
It consists of schooling and training on the job as concurrent activities. Trainees learn basic information or 
skills in the classroom and the practical implementation while working for associated companies. In general, 
this dual system of vocational education is regarded as a very effective way to train young people. However, 
there is potential for improvement. Current disadvantages of the dual system of vocational education are 
[BIB01]: 

• Limited technological infrastructure of small companies. 

• Different points of time for the transmission of specific knowledge from the curricula in schooling 
and job training. 

• Lack of communication between the teacher of a vocational school and business companies. 
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Using e-learning tools supports trainees and teachers at the same time. Furthermore, these tools help to 
improve the named disadvantages above. TPL offers the possibility to collect data of individual learning 
rates. Further, TPL supports the transmission of knowledge across different locations and learning times 
[TPL14]. Basic information like the basic functions of a computer-aided-design (CAD) program, for 
example, can be taught in a new way with tutorials, practices and self-learning modules. Incidentally, trainees 
and teachers can create their own learning content as well as specific learning modules. During the project 
TPL will provide learning material specialized for technical product designer. 


2. CURRENT STATUS OF THE PROJECT 

The aim of an initial survey stage is to identify topics and available educational material in cooperation with 
the vocational schools and business companies. The result of this project’s stage is to develop a catalogue of 
requirements about how to develop a technical structure, didactical content design and content topics. 

2.1 Applied Survey Methods 

Professional training in Germany is based on (national) training regulations for vocational schools and 
business companies. Within the first stage, training regulations are analysed to detect suitable topics for 
e-learning. During the second step, interviews of teachers and business-instructors are conducted to gather 
information about didactical ideas and proposals for the platform’s technical structure. 

2.2 Results of the Training Regulation Analysis 

The training regulations for the technical product designer consist of 18 learning sections. The two learning 
sections shown in Figure 1 are part of the training content for the first and second year. By doing a survey 
with the teachers about the training regulation for vocational schools the results show that 100% of the 
learning topics within the first learning section are suitable for e-learning modules. The survey additionally 
results showed also that the fourth section contains only 75% of suitable topics for e-learning materials. 


Suitability content for E-Learning (exemplary selection: learning section 1 and 4) 


§ 9 

| ' | 









learning section 1: Analyze and report technical systems 


■ 









learning section 4: Execute orders customer-oriented |jfg 










0,00% 10,00% 20,00% 30,00% 40,00% 50,00% 60,00% 70,00% 80,00% 90,00% 100,00% 

Figure 1 . Training regulations for vocational training, learning section 1 and 4 

During the first learning section trainees learn basic tasks and skills, for example free-hand drawing or the 
usage of CAD-systems. These are basically single handed learning tasks with a PC. In contrast, learning 
section 4 requires teamwork while working on a customer order. At the end of this section the trainees have 
to compare their results with the results of other teams. The necessity of intensive personal communication is 
the reason that this learning section has a lesser e-learning potential. 
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The results indicate that all 18 learning sections are similar. 15 learning sections were assessed as 100% 
suitable of developing e-learning content for their topics. Main topics of the developed modules are: 
engineering drawing and documentation, virtual construction, materials science, manufacturing processes and 
project-quality management. 

2.3 Structured Interview Analysis’ Results 

The analysis of structured interviews pointed out that teachers and business-instructors recommend content 
oriented developing during the working process. Therefore, didactic conception of teaching and learning 
modules will challenge trainees in collecting self-reliant information, planning the execution, create a product 
and checking results. According to specific topics of the learning sections, new material includes learning 
content, learning objectives, exercises, tests and also provides a learning video. To summarize, the didactical 
conception of all materials and the design of the user interface support self-organized learning. 

The communication structures of TPL have to ensure a secure and easy way of communication between 
teachers, business-instructors and trainees of all participating business companies in all directions. A 
multidirectional communication tool will be developed. 

Three internal working spaces, one for the vocational schools, the business companies and trainees, will 
be provided by the e-learning platform. The working spaces of vocational schools and companies create the 
possibility of working with the trainees at internal topics, exercises or test. The working space for teachers 
and business-instructors provides functions for developing new teaching learning materials and store internal 
documents. 

The working space for the trainees contains the self-learning materials, the possibility to establish 
learning groups between trainees, a glossary and a link collection to support the learning processes. The 
following section describes the further steps of the development. 


3. NEXT STEPS 

The next steps during 2014 include the realisation and implementation of the platform. Generating learning 
content for technical product designers and taking account of the didactical concepts is also planned. 

3.1 Designing the Web-Based Platform 

Developing the platform includes technological realisation and creating its content. The first stage consists of 
designing a platform and its implementation. After installing basic elements, e-learning modules will be 
added to the platform. This implies a parallel execution of developing the web-based platform and creating 
the content. 

Learning and teaching material should be provided in hypermedia, which are different kinds of media 
formats. This allows teachers, business-instructors and trainees to work with the content on- and offline. The 
content is presented directly on the platform in a PDF file and as a presentation; this will be supported by 
videos. The modules will also include different tests to control the individual level of knowledge. 

Therefore, TPL uses an open-source learning platform “Moodle” [TPL14]. “Moodle” can completely be 
adapted to the demands of a given project and area of work, for example, the different working spaces for 
teachers, instructors and trainees. With the user’s help of a role system “Moodle” users have the opportunity 
to decide which information, topics and the amount of detail are visible to other user. Furthermore, it 
supports self-organized learning like basic information of gradually explaining an engineering drawing 
[TPL14]. For example, a learning module starts with an introduction about free-style drawing, then sets out 
line widths and completes with standards [HeLM14.1], Additionally “Moodle” supports different forms of 
tests like multiple choices tests, free text questions or self-learning packages. “Moodle” evaluates the used 
learning modules automatically. The results of an evaluation will be used within the project to improve the 
developed platform. 
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3.2 Developing Teaching and Learning Content 

Preparing content for the dual training of technical product designers is the key task of the project. The next 
step before developing learning and teaching content is to gather learning and teaching materials, which are 
used in vocational schools and business-companies. The dual vocational education is subdivided into stages, 
which could be reused as a general procedure for the training. The materials on the platform will be arranged 
to this periods. Using the didactical concept as a basis the collected teaching and learning content can be 
rewritten and used to create new content. Vocational schools and business-companies are involved during 
this step. Business companies assess the new materials and support the development of material for future 
users. The results of this assessment will be evaluated and used to revise the platform. 


4. CONCLUSIONS 

The results of the first project stage will be used to build a concept for a web -based platform and to realize it 
within TPL during the rest of the projects period [TPL14]. The next months will be used to create target- 
group-specific e-learning content which should be used to support lessons during schooling and job training. 
Realising a digital and modern learning place is the main objective of the project TPL [TPL14]. The designed 
platform will consider the results of interviews made with vocational teachers and business companies so that 
the platform will be designed as practical as possible for all users. 
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ABSTRACT 

This paper reports on an investigation of the use of Facebook as a tool to mediate learning amongst distance learners at 
Makerere University, a dual-mode institution offering both conventional and distance learning programs. While 
conventional courses take 17 weeks in a semester, the distance learners come in for two residential sessions, each lasting 
for two weeks. The study focused on Bachelor of Commerce (External) students doing an Introduction to Information and 
Communication Technology course. This course was run through face-to-face sessions and the use of Facebook to 
increase interaction among learners. Out of the 650 students on the course, 621 joined the Facebook class page. In this 
paper we report on the students’ perceptions about using Facebook for teaching and learning, and specifically if it 
mediated interactive learning. The Facebook class page was used as a discussion board. The questionnaire on which this 
evaluation was based was uploaded two months after the course ended. Interactive feedback through posts. Tikes’ and 
comments were received from students. Results show that learners appreciated the use of Facebook for interactive 
learning, hence fostering meaningful learning. We also tried to match the affordances of Facebook for modeling, 
contingency management, feedback, instructing and questioning. Feedback and questioning were well received on 
Facebook during the course, indicating that Facebook mediated interaction and learning. We contend that if Facebook is 
to be used for meaningful teaching and learning, then there must be an appropriate and systematic mechanism for 
integrating educational activities in the platform. 


KEYWORDS 

Facebook, Distance Learning, Mediation, Interactive Learning 


1. INTRODUCTION 

World over Distance Learning has been presented as a viable alternative to traditional methods of teaching 
and learning for addressing the massive demand for higher education (Mayende, Divitini & Haugalokken, 
2006). Uganda introduced free universal primary and secondary education in 1997 and 2007 respectively. 
This has led to increased demand for university education (Aguti, 2002). According to Aguti and Fraser 
(2006), in distance learning, the learner spends most of his/her time away from the university studying on 
their own using learning materials which are either print, electronic or both. Makerere University distance 
learners come to the University for residential sessions twice a semester. Most of the remaining time, they are 
at their work places or homes. Distance learning at Makerere University is still in the first generation, relying 
on print materials and occasional face-to face sessions. Students are distributed across the country in places 
where access to electricity and Internet is a problem (Aguti & Fraser, 2006). The distance learning materials 
are designed to be interactive; that is, they act as the teacher to the learner, thus enhancing independent 
learning (Oliver & Herrington, 2003). However, even if distance learners have well developed learning 
materials, they require additional support away from the University. Various distance education practitioners 
have advocated for the use of ICTs in the teaching and learning (Aguti & Fraser, 2006; Muyinda et al, 2010; 
Kajumbula, 2006). They have for instance advocated for and undertaken research in the use of mobile phones 
for teaching and learning on distance learning programs (Muyinda et al, 2010). 
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Facebook is a social networking site that allows students to interact with one another informally (Rambe 
& Ng’ambi, 2011). However, little research has been done to find out if Facebook can be used as a tool to 
mediate learning. This paper investigates how Facebook mediated learning amongst distance learners at 
Makerere University. Mediation, according to Vygotsky (1978), is the guidance given to the learner to 
acquire unique human knowledge or moving a learner from their current knowledge level to the point of 
knowing. 


2. LITERATURE REVIEW 

2.1 Inter and Intra-Psychology and Facebook 

This paper explores the use of Facebook for interactive learning. Continuous interaction among learners 
breeds inter- and intra-psychological learning (Vygotsky, 1978). Inter-psychology begins as a social 
interaction between individuals and, as this interaction goes on, it is internalized hence intra-psychology 
(ibid). When knowledge is internalized as a result of continuous interaction, it becomes part of us. In order to 
get to intra -psychology one must be guided/assisted (Vygotsky, 1978). Since intra-psychology arises from 
inter-psychology, which begins with social interaction, therefore, social networking sites like Facebook can 
afford or mediate inter- and intra-psychology. 

2.2 Mediation of Inter- and Intra-psychology 

Mediation is a fundamental concept of human development. Mediation can be defined as a process of 
supporting a learner to learn new concepts by the tutor or by more capable peers. Facebook comes in handy 
in mediating the interaction and learning among distance learners, hence potentially leading to inter- and 
intra-psychology. Mediation can guide the learner to acquire his/her accessible knowledge, which we refer to 
as the zone of proximal development (ZPD). Vygotsky (1978) believes that language is the primary 
mediator. However, Gallimore and Tharp (2002) have argued that, it is not only language that develops the 
mental cognition. They proposed five means of assistance to learning: modeling, contingency management, 
feedback, instructing, and questioning. Our analysis of the affordances of Facebook as a teaching tool is 
based on this framework. 

Modeling is a process of assisting through imitation. Usually learners (most especially young learners) 
learn by imitating. Although it is clear that modeling can assist learning, Facebook does not fully support 
modeling as a mode of assisting learning. However, in circumstance where pictures and videos are uploaded 
on the Facebook class page, learners can learn through imitation. 

Contingency management “is the means of assisting performance by which rewards and punishments are 
arranged to follow behavior, depending on whether the behavior is desired or not.” (Gallimore & Tharp, 
2002: 179). Gallimore and Tharp (2002) further assert that effective teaching and learning depends more on 
positive rather than negative rewards. That is why we are encouraged to use only positive rewards. However, 
since contingency management does not introduce new behavior, a teacher who is interested in introducing 
new behavior should use other methods of assistance. Contingency management on Facebook can be 
supported by positive comments and by using the Tike’ functionality. Teachers or students can Tike’ each 
other’s comment or post, thus giving positive feedback. Likewise, they may share positive and constructive 
comments. 

Feedback is the process of responding to what someone else has done. To Gallimore and Tharp (2002: 
180), “[s]imply providing performance information is not feedback; there will be no performance assistance 
unless the information provided is compared to some standard”. In distance learning, instructors are 
encouraged to give feedback on assignments to help learners improve their learning. When using Facebook, 
teachers should avoid giving performance information that does not assist the learner. 

Instructing is the process of assisting the learner through telling him/her what is supposed to be done. 
Instructing is a behaviorist approach to learning (Saul and Patti, 2008). Instructing is still a common practice 
with tutors who deal with large classes like those on the distance learning programs at Makerere University. 
For instruction to be effective, it must be blended with other methods of assisting learning, like contingency 
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management and feedback, where applicable. Although Facebook does afford for instructing, we did not use 
it much for instructing on this project. 

Questioning is a very important method that assists performance although many teachers confuse 
assessment questions with assisting questions. It should be noted that not all questions assist performance. 
Assessment questions are aimed at finding out the ability of the learner to perform without assistance while 
assistance questions are used to provoke the thinking of the learner to the level the learner would not have 
attained by himself/ herself. If the tutor understands how to question, then Facebook offers that possibility. 

The discussion above has elucidated that learning can be teacher or fellow learner or technology 
mediated. The rest of this paper zeroes in on technology (Facebook) mediated learning. 

2.3 Technology Mediated Learning 

Technology mediated learning refers to using technology to assist learning (Oliver & Herrington, 2003). 
Recently, there has been increased use of new technologies in teaching and learning (Daina Laurillard, 2006). 
Collis (1977) cited in Oliver and Herrington (2003) argues that when using technology for learning, there is 
need to re-engineer the entire learning design rather than just re -packaging content electronically. Usually, 
we design learning environments that mimic the teacher in a traditional class (Oliver & Herrington, (2003). 
This is common in the distance learning classes because of the large class sizes. Oliver and Herrington (2003) 
have suggested three important components for improving online learning design, namely: tasks to engage 
and direct the learner, supports for the online learner, and the learning resources. They also believe that we 
should have authentic tasks which help learners to learn so that the learner mimics the real working 
environment. They indicate that “flexible and online learning environments need learning supports to be 
designed as integral parts of the learning process” (p. 14). They further contend that when dealing with a 
learning support component, there is need to work carefully on the supportive tutor role which should 
provide learners with feedback and guidance on requests and most especially at critical points in the learning 
process. These are affordances inherent in Facebook since it can be ported on multiple platforms, including 
mobile phones. 

2.4 Facebook as a Mediator of Learning 

Vygotsky (1978) argues that social interactions are seen to play a critical role in the processes of learning and 
cognition. We also realize that learning support can help mediate learning through providing guidance and 
feedback to the learners (Oliver & Herrington, 2003). Facebook can ensure that guidance and feedback is 
achieved. Rambe and Ng’ambi (201 1) believe that the use of Facebook reinforces the rights to inform and be 
informed and such interaction leads to rich knowledge resources based on multiple voices. Vygotsky (1978) 
says that development can be achieved through the collective nature of human consciousness manifested 
through social interaction. Rambe and Ng’ambi (2011) identified the need for the active presence of the 
learners and tutors on social media for the learners to have someone to interact with at any time. This can 
then help reduce the feeling of isolation characteristic of distance learning. 

There are also challenges involved when using Facebook. It is estimated that online teachers spend 90% 
of their planning and development time creating content and online learning resources (Oliver & Herrington, 
2003). Mediation is the guidance that goes beyond online content. Rambe and Ng’ambi (2011) identified 
challenges of using social networking sites as redundant postings, limitations of collective responsibility, 
subtle negotiations of power between educators and learners, and confusion of roles among novice learners. 
They observed that some of the challenges of using Facebook as a pedagogical tool are both the educator 
gaining access to Facebook spaces of all learners and ensuring that every member of the class is a friend of 
everyone. They argue that Facebook violates personal privacy, and some students may be skeptical about 
participating in academic activities that invade their personal lives and “comfort zones”. 

Different scholars have suggested the use of different activities in online learning design. Online learning 
mainly supports activities designed to involve peer cooperation and collaboration (Oliver & Herrington, 
2003). This is eminent in social networking sites like Facebook. Anderson (2003) argued that if any of the 
interactions, namely, teacher-to-learner, learner-to-learner and learner-to-content are increased, then 
meaningful learning can take place. Rambe and Ng’ambi (2011) found out that Facebook could be accessed 
anywhere, anytime by students using mobile phones. This is also evident in the research done by Muyinda 
and Mayende (2013), which showed that the majority comments and posts on the class Facebook page were 
from mobile phones. 
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3. MATERIALS AND METHODS 

Facebook was used in conducting the Introduction to ICT course taught to third year Bachelor of Commerce 
(External) students at Makerere University. This course was run in the second semester of academic year 
2012/2013. The class capacity was 650 students but only 621 joined the Facebook page. The course ended in 
May 2013 with the students sitting end-of-semester examinations. Two month later, we posted a qualitative 
questionnaire on the Facebook class page to find out how students felt about Facebook as a tool to mediate 
interactive learning. The questionnaire and the comments were read by every member who visited the page. 
This provided interactive feedback and verification of the students’ comments. We also interviewed two 
students to get a better insight into how students used Facebook and how it could mediate learning. These 
students were randomly chosen from the students who had responded to the questionnaire. 


4. FINDINGS 

Most of the posts on the page vividly indicated that Facebook as tool for mediating learning could not replace 
the teacher. One of the comments read, “Seeing a teacher in class is more important, practical and effective. 
The explanations were better understood than on Facebook.” Students provided various reasons to support 
their stance. For instance, some students felt that it was difficult to explain complex concepts using 
Facebook. Other students felt more confident listening to a human teacher than reading Facebook text and 
most especially when the majority of text was coming from their peers. Student’s numbers were very big as 
mentioned earlier. It was therefore difficult for the course facilitators to read the different posts and comment 
on each one of them. Also Facebook has no affordance of enabling teachers and learners see each other’s 
facial expressions. Facial expressions sometimes help the learners to comprehend complex concepts. It can 
also help the teacher to read from learners’ faces whether or not they are confused. In support of this 
imperative, one of the students said, “Seeing a teacher in class is more important, practical and effective”. 
Explanations were better understood in the physical classroom than on Facebook. This also relates to another 
post which indicated that the gestures made by the teacher during the face-to-face sessions, caused the 
students to understand better than on Facebook. Students preferred discussing their minds with their teacher 
face-to-face than on Facebook. Some students felt that their Facebook page had too much text and required 
them to read all the time. Ideally, this means that despite the affordances Facebook offers, students did not 
enjoy reading too much and, sometimes, repeated text from multiple learners. 

With the above negativity, one would be left to wonder whether Facebook mediated learning at all. The 
negativity was attributed to the newness of this mode of support. It was the first time learners were using 
Facebook for learning support. It is hoped that as time goes by, learners’ perceptions could change. Already, 
numerous positive perceptions were registered. 

Some students felt that Facebook was very important and they explained how it mediated learning. 
“Facebook gave me access to lecture notes through the upload functionality on the Facebook group page,” 
said one of the students. Another said, “Facebook gave me the opportunity to access the class from anywhere 
and anytime via my mobile phone”. Another said, “Facebook gave me opportunity to learn while out of class 
and in a more convenient way”. 

Some students suggested that both face-to-face and Facebook are vital and can complement each other. 
The teachers were able to find out areas where students were finding problems so as to give further 
explanation during residential sessions. So Facebook helped teachers to know the ZPD of their learners. The 
students also said that Facebook was good for interaction. This interaction is what Vygotsky (1978) says 
begins as inter-psychology and leads to intra-psychology when it is internalized. Anderson (2003) argues that 
if there is increased interaction either between learners and learners or between teachers and learners, 
meaningful learning occurs. This shows that Facebook as a tool can mediate learning since it afforded 
students the possibility to interact. Students shared and discussed ideas with teachers and with their peers. 
Students also said that Facebook helped them to be guided on how to do certain things in the right way by 
asking their peers or the teacher. This suggests that guiding learners caused them to learn, hence Facebook 
mediated learning. 
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Students also reported that the teacher was able to ask them questions on Facebook and the learners 
responded, which created interaction. Gallimore and Tharp (2002) say that questioning is one way of 
assisting learning. Students felt that prompt answers made by the teacher made the learners learn better. 
Feedback is a key aspect in assisting learning. However the type of feedback matters as mentioned earlier. 
Students felt that Facebook helped them do revision. This means that they found content which could be 
discussed on the class page. Content is vital to the learning process. By providing access to content, 
Facebook mediated learning. Students said that some of the content discussed on Facebook appeared in the 
final examinations, meaning that the content was relevant and the students learnt it. Students said that 
Facebook enabled them to read widely; whenever they found new concepts of Facebook, they researched 
further into them. Therefore, it is clear that Facebook mediated learning through questioning that made the 
students look for more information. Students also felt that Facebook was vital for the students who did not 
attend face-to-face sessions. Students felt that Facebook enabled them to receive frequent updates about the 
course. This administrative support is equally important for learning. Students also appreciated the 
experience of using Facebook in the class room environment. They also gained confidence in using 
computers during the course unit since some of them had never used computers for learning before. 

In using Facebook for mediating learning, several challenges were registered. Some students indicated 
that they failed to download the notes which were uploaded on Facebook. This was common because the 
majority of students were using mobile phones some of which did not support the download function. Also 
some students did not know how to download content from Facebook, which calls for basic training of the 
learners in using Facebook before the course starts. Another challenge related to limited access to Internet- 
ready computers, hence limited access to Facebook. Some of the students had access to the computers at the 
main campus; but these too were too many for the few computers at their disposal. Students said that Internet 
access was a problem and hindered their usage of Facebook. Given that there is a cost attached to accessing 
Internet both through mobile and computers, students found a problem accessing Facebook. Some students 
also said that they found problems with mobile network coverage when they travelled. Also, some students 
did not have Facebook accounts and others had mobile phones which did not have capability to access 
Facebook. Such learners were thus disadvantaged. 

Students’ suggestions for improving Facebook usage in teaching included: providing constant feedback 
from the teacher so that the students continue to be engaged and guided in the process of their learning, and 
increasing participation of the student peers so as to increase interaction. 


5. CONCLUSION 

The possibility accessing Facebook on mobile phones gave distance learners an opportunity to access the 
class. Given that the majority of the distance learners have no access to computers and Internet from their 
places of aboard, mobile phone access to Facebook offers the best alternative. The use of Facebook in the 
teaching and learning helped to support modeling, contingency management, feedback, instructing and 
questioning. Modeling and instructing could be supported through videos uploaded on the class page. For 
contingency management, students felt that they were not sufficiently rewarded with positive comments. 
Facebook allowed students to easily give feedback to each other through ‘liking’ peers’ comments. 
Facilitators were called upon to increase interaction and give timely feedback to comments. Students also 
said that Facebook helped them to get guidance. Facebook supported feedback as mentioned in the results. 
Therefore, Facebook mediated learning through feedback and questioning. It was established that questioning 
was possible through Facebook but teachers were cautioned to be careful not to confuse assessment questions 
with assisting questions. There is need to systematically integrate the activities in the teaching and learning 
using Facebook in order to achieve meaningful mediated learning. 
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ABSTRACT 

In this study we surveyed students in a mid-sized university in Ontario, Canada to explore various aspects associated with 
their use of computer-based applications. For the purpose of analysis, the computer applications under study were 
categorized according to the Human-Computer-Human Interaction (HCHI) model of Desjardins (2005) in which 
interactions between users and digital technology are categorized into four classes of interaction, namely. Technical 
Interactions (interactions with the digital devices themselves), Social Interactions (interactions with other users through 
digital devices), Informational Interactions (interactions with information through digital devices), and Computational 
Interactions (interactions with data processing software through digital devices). The survey attempted to assess the 
following four aspects of computer application use (in the context of the HCHI model): importance, purpose, frequency, 
and confidence. In this paper we report on preliminary findings regarding the purpose and importance students attributed 
to the applications under study. Frequency and confidence studies were reported elsewhere — Partosoedarso, DiGiuseppe, 
vanOostveen, & Desjardins (2013). Preliminary findings indicate that, in general, students in this study tended to engage 
in technical, social, and informational interactions primarily for personal purposes and computational interactions for 
school purposes. In terms of importance, students ascribed the greatest importance to social interactions, followed by 
technical, informational, and computational interactions, in that order. 


KEYWORDS 

Information Communication Technology, higher education, digital technology 


1. INTRODUCTION 

Information and communication technology (ICT) is rapidly becoming a pervasive and highly valued 
resource in all endeavours, including post-secondary education. Laptops and mobile devices, including 
smartphones, tablets, which enable ubiquitous Web surfing, video conferencing, and social networking have 
for some time now been deemed necessary tools in higher education (Melton & Kendall, 2012). Furthermore, 
highly interactive — and highly motivating — applications such as serious games, augmented reality, and 
massively open online courses (MOOCs) are making greater inroads into post-secondary schooling (Dede, 
2013; Johnson, Smith, Willis, Levine, & Haywood, 2011). Almost three-quarters of these institutions offer 
online courses and almost one quarter of all post-secondary students having taken at least one online course 
in 2011 (Parker, Lenhart, & Moore, 2011). Currently, most North American post-secondary institutions offer 
online courses in which geographically distributed students and instructors learn together in virtual 
classrooms. These developments have placed great pressure on students and instructors to keep up with the 
trends, assess the value of these new technologies, determine their most appropriate uses, and develop 
essential skills. 

It can be assumed that university students are likely exposed to a variety of computer-based devices and 
applications in their personal lives, work lives (if gainfully employed), and studies, especially given the 
pervasiveness of ICT in today’s (Canadian) educational milieu. It may also be assumed that as students gain 
experience with various devices and applications, they will gain confidence in their use and will likely begin 
to find some devices and applications more useful than others for various purposes. In other words, they may 
attribute varying levels of importance (i.e., value) (Wyse, 2011) to the devices and applications they use. 
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Thus, in the broader study on which this paper is based, we assessed — via an online survey — the following 
four aspects of computer-based device/application use: frequency, confidence, purpose, and importance. In 
earlier papers (DiGiuseppe, Partosoedarso, vanOostveen, & Desjardins, 2013; Partosoedarso, DiGiuseppe, 
vanOostveen, & Desjardins, 2013), we reported on findings regarding the frequency with which a group of 
university students employed a particular inventory of common devices and applications, and the level of 
confidence they developed in using those technologies. In this paper, we extend the findings of that study by 
reporting our findings on the purpose and importance students attributed to the same inventory of computer- 
based applications discussed in the papers noted above. 

1.1 Frameworks 

In this study, we employed a survey instrument adapted from Desjardins & Bullock (2012) in which Human- 
Computer-Human interactions (HCH interactions) are categorized into the following four types: Technical 
Interactions (TI) (users interact with digital devices for basic operations). Social Interactions (SI) (users 
interact with digital devices for social purposes). Informational Interactions (II) (users interact with digital 
devices for information), and Computational Interactions ( Cl) (users interact with digital devices for creative 
and computational purposes). Table 1 provides some examples of these four types of interaction. 


Table 1 . HCH Interactions (adapted from Desjardins & Bullock (2012)) 


INTERACTION 

DESCRIPTION 

Technical (TI) 

Employ basic devices and applications; Create/edit documents/multimedia 

Social (SI) 

Communicate via email, video, text, audio; Use social networking systems 

Informational (II) 

Search for and exchange articles, video, music, books, etc. 

Computational (Cl) 

Create concept maps, plans, diagrams, pictorials, graphics; Process data 


Survey responses provided by the participants in this study regarding their interactions with ICT were 
analyzed in relation to the four types of interactions of the HCHI Model summarized in Table 1. 


2. METHODS 

In this study, we surveyed a sample of university students in a mid-sized university in the province of 
Ontario, Canada to assess their use of a variety of current computer-based technologies. The survey 
questionnaire employed three Likert scales for assessing “Importance of Use,” “Frequency of Use,” and 
“Confidence of Use,” and a nominal scale to assess “Purpose of Use” (Table 2). 


Table 2. Scales used to assess importance of, purpose of, frequency of, and confidence in using computer-based 

technology 


Importance of use 

Purpose of Use 

Frequency of use 

Confidence of use 

1 = not important 

2 = somewhat important 

3 = very important 

4 = indispensable 

personal 

studies 

work 

1 = never 

2 = few times a year 

3 = few times a month 

4 = few times a week 

5 = daily 

1 = do not know how to use 

2 = not confident 

3 = confident 

4 = quite confident 

5 = very confident 


A total of 423 students (n = 423) were invited by email to complete the online survey in February-March, 
2013 and 157 students (n = 157) voluntarily completed the survey, for a 37% completion rate. The segment 
of data pertinent to this paper were analyzed in Microsoft Excel using descriptive statistics for students’ self- 
assessments of the purpose for which the students used various devices and applications and the importance 
they attributed to those technologies. Survey responses were then coded, categorized, and analyzed in 
relation to the four types of HCH interactions of the HCHI Model summarized in Table 1. 
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3. FINDINGS AND DISCUSSION 
3.1 Purpose for using ICT Applications 

Figure 1 indicates that, in general, students in this study most often employed ICT applications involving 
technical, social, and informational interactions (Desjardin, 2005) for personal purposes. This is 
demonstrated by the relatively high proportion of responses indicating personal use of social interactions 
such as texting, audio recordings, social media, and sharing ideas with others. 


I personal 


I studies Gwork 



Computational 


Figure 1 . Purpose for using various computer applications. 

With the exception of emailing (a social interaction) and sharing calendar information, sorting data, 
producing graphs, and performing complex calculations (computational interactions), very small proportions 
of participating students employed most of the other applications assessed in this study for work-related 
purposes. This may be an indication that small numbers of the students had jobs, or if employed, their jobs 
did not require significant use of these other applications. However, the chart also indicates that more equal 
proportions of students employed email — a principal means of communication globally (Robinson, 2012) — 
for personal, work, and study purposes. This suggests that while emailing was not as popular a way of 
interacting socially for personal purposes as texting, audio, video, and social media were, significant 
proportions of students indicated that they used email for all three purposes assessed in this study (personal, 
work, and studies), further suggesting that these students considered email to be a more versatile and more 
broadly applicable form of social interaction. 

Figure 1 also shows that a majority of students engaged in a minority of technical, social, and 
informational interactions for study purposes. For example, only editing documents (a technical interaction), 
sharing documents (a social interaction), and searching for articles (an informational interaction) were 
employed by a majority of students for study purposes. However, a majority of students engaged in most of 
the computational interactions for study purposes, not personal or work-related reasons. Computational 
interactions include activities such as concept mapping, sorting and processing data, graphing, and 
calculating. These findings seem to align with the sorts of applications students commonly employ in 
completing school assignments, especially in contemporary universities in which the use of ICT is becoming 
increasingly pervasive (Lai, 2011). Interestingly, however, is the generally low proportions of students who 
indicated use of social interactions such as video, texting, audio, and social media, and the very limited use of 
some of the informational interactions, such as searching for music and movies, for educational (study) 
purposes — findings consistent with those of Selwin (2007) who maintains that “many university students and 
faculty make only limited formal academic use of computer technology .... use of computer technologies in 
many areas of higher education could best be described as sporadic, uneven, and often ‘low level” (p. 84). 
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These findings seem to suggest that highly social interactions (video, texting, audio, social media) and 
informational interactions with social affinities (searching for movies and music) may still not be highly 
integrated in post-secondary teaching and learning as they could be, suggesting further research in this area. 

3.2 Importance attributed to Various ICT Applications 

Figure 2 is a bar graph illustrating the levels of importance participating students attributed to the various 
computer interactions studied in this project. 


■ Indispensable EJ Very important 83 Somewhat important ■ Not important 



Figure 2. Level of importance students attribute to various computer applications. 

Some interesting patterns may be discerned in Figure 2. For example, within the category of 
computational interactions, a majority of students attributed relatively low importance to all of the 
applications in this category, including concept mapping, sorting data, graphing, and performing complex 
calculations — with the least importance ascribed to concept mapping. When comparing these results to the 
purposes described in Figure 1, it is obvious that the majority of students claimed to have used these 
applications for study purposes, not for work or personal reasons. Within the social and informational 
interaction categories, the importance ascribed to the various applications is highly variable; however, some 
interesting results are noticeable. For example, a very large proportion of students valued email very 
highly — an application which the results in Table 1 showed was employed by approximately equal numbers 
of students for personal, academic, and work related purposes. This tends to indicate the importance and 
pervasiveness of email in virtually all aspects of these students’ lives. 

Another notable result is that within the category of technical interactions, a very large proportion of 
students considered editing documents (an application most students performed for educational purposes) to 
be a very important activity while editing voice recordings and multimedia (applications most students 
performed for personal reasons) not to be so important. A similar pattern is evident within the informational 
interaction category where searching for articles is highly valued by a majority of participating students and 
used by a majority for educational purposes while searching for video clips, movies, music, and e-books are 
not highly valued by a majority of the students and used by a majority for personal purposes. This seems to 
indicate that in a number of cases, participating students valued applications used in their studies more highly 
than applications employed primarily for personal purposes. This may simply be an indication that the 
participants were students at the time the survey was conducted, and, as such, more of them tended to value 
applications required for achieving success in their studies more than those used in their personal and/or work 
worlds. Nevertheless, this was not a consistent finding. For example, texting (a highly valued social 
interaction) was employed by a large majority of students for personal purposes while sharing documents (a 
moderately valued social interaction) was employed by most students for educational (study) purposes. 
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4. IMPLICATIONS AND CONCLUSIONS 

The small number of analyses reported in this brief paper tend to indicate that in many of the technical, 
social, and informational HCH interactions assessed in this study, participating university students tended to 
value more highly those ICT applications employed in their studies more so than those employed for work or 
personal purposes. However, this was not a consistent finding, for there were cases in which the reverse was 
true. In general, however, it was found that none of the interactions (i.e., applications) assessed were used for 
the purpose of employment (i.e., work). This, however, was likely a reflection that only a small proportion of 
the participating students held a job at all, or one that required them to use the assessed applications. In terms 
of the computational category, most students employed these applications for educational purposes, but did 
not value them highly in large numbers. Interestingly, in a paper on another aspect of this study, 
Partosoedarso, DiGiuseppe, vanOostveen, and Desjardins (2013) reported on the “frequency” and 
“confidence” with which students used these computational applications, and indicated that the students 
tended to use these applications infrequently, and possessed relatively little confidence in their use. Taken 
together, these and the current findings suggest the need for more research in this area, including qualitative 
case studies to help determine why students tend to value such potentially academically useful ICT 
applications so low; why they tend to use them so infrequently; and why they tend to have such little 
confidence in their use. 
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ABSTRACT 

This paper proposes a novel e-Learning system for the comprehensive understanding of electronic circuits with 
transistors. The proposed e-Learning system allows users to learn a wide range of topics, encompassing circuit theories, 
design, construction, and measurement. Given the fact that the amplifiers with transistors are an integral part of almost all 
the electronic equipments, a fundamental knowledge of the design and construction of transistor circuits is highly 
imperative in the field of technology education. To this end. the proposed system serves as an effective educational tool 
for learning practical electronic circuits. The usefulness and effectiveness of the proposed system were evaluated by 10 
university students in an actual class. The positive responses provided by all the students indicate the usefulness of the 
proposed system. 


KEYWORDS 

Electronic circuit, bipolar transistor, e-Learning, circuit construction, SPICE simulation 


1. INTRODUCTION 

Teaching and learning electronic circuits are important elements in the field of technology education. 
Recently, several education systems have been developed to improve the students’ understanding of the 
concepts of electronic circuits. These so-called e-Learning systems for electrical circuit analysis (Weyten 
et.al, 2009) and the circuit design of the amplifier with a bipolar transistor (Assaad et. al, 2009) facilitate 
understanding of the fundamental theories and analysis of simple circuits. A learning kit to help beginners 
understand the functioning of various components in the electrical circuit was developed (Reisslein et. al, 
2013). However, these conventional systems are based on all-purpose or ready-made learning tools and are 
suitable for only specific circuits within a subject area. Moreover, these systems are not designed to promote 
the learning of circuit construction and experiments, although the ability to understand and construct 
electronic circuits is highly imperative for acquiring extensive knowledge of the technologies. The 
educational support system, which was proposed to overcome the disadvantages of the conventional learning 
tools, involves experiments on the circuit construction of an active filter and a simple sound processor with 
operational amplifiers (Takemura, 2013). However, the abovementioned system has the following 
disadvantages: 

• The system is suitable for constructing only simple circuits with operational amplifiers. 

• It is necessary to provide comprehensive studies for the users to acquire deeper insights on the 
process. 

In general, understanding of transistor circuits is difficult for beginners, as the designing of the circuit 
demands knowledge of complicated theories and analysis. Given the fact that the amplifier with a transistor is 
an integral part of almost all electronic equipments, a comprehensive understanding of transistor circuits is 
essential in the field of technology education. To this end, this paper aims to develop an e-Learning system 
that is capable of providing comprehensive studies on electronic circuits in terms of (a) circuit theories, (b) 
analysis, (c) design, (d) construction, (e) measurement, and (f) evaluation. 
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2. TECHNICAL FEATURES OF THE PROPOSED SYSTEM 


Figure 1 shows the schematic illustration of the proposed e-Learning system that aims to provide 
comprehensive studies in the aspects of (a)-(f) described in Section 1. This system consists of individual 
computers of the users and a remote education system that runs online. The individual learners in the 
network learn the comprehensive topics of circuits described in Subsections 2.1 and 2.2. 

2.1 Web-based Learning of Circuit Theories and Design 

This section describes the new functionality of the proposed system that performs an important role in the 
designing the construction of the circuit. First, learners download the manuals and the datasheets necessary 
for understanding the theories and analysis methods for designing and constructing circuits. The datasheets of 
the circuit devices, which are necessary to grasp the characteristics and rated values of devices, are provided 
by the manufacturers. Based on the required circuit design, the learner uploads the details of the designed 
circuit (e.g., the components used in the circuit and calculated parameters) to the remote education system. 
The remote education system evaluates the designed circuit and specifies the errors in the circuit by 
comparing with the details and specifications of the designed circuit prepared by an instructor. 


2.2 Circuit Construction and Experiments 


Based on the circuit design using the technique described in Subsection 2.1, the learner constructs the 
actual circuit on a breadboard and transmits an image of the constructed circuit to the remote education 
system. The remote education system performs image processing to recognize the circuit construction and 
translates its structure into a general circuit description language (Simulation Program with Integrated 
Circuit Emphasis; SPICE) (Rabaey, 2012). This SPICE translation technique simulates the working of the 
circuit and enables virtual measurement without real measurement equipment. Furthermore, it can also 
identify the incorrect components in the circuit (Takemura, 2013). Moreover, the circuit translation and 
simulation techniques adopted in this system are important for improving the efficiency of experiments 
and preventing the occurrence of serious accidents, such as electric shocks and fires. Depending on the 
required purpose and environment (e.g., virtual laboratories and e-Learning), individual learners can 
choose a preferred mode from the following learning modes; (1) virtual circuit making (VCM), (2) 
real circuit making (RCM), and (3) the mixed mode (Takemura, 2013). To construct the virtual circuits 
with transistors using VCM, this study added new virtual circuit components to the database of the remote 
education system. In addition to these features that are available in the preceding systems, the proposed 
system includes new functionalities for performing web-based experiments using virtual measurements. 
Using this function, a user can learn the characteristics and the working of the constructed circuit. The 
remote education system also evaluates the characteristics of the circuit constructed by the user, such as the 
amplification ratio and frequency bandwidth. 
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3. METHODOLOGY FOR EVALUATING THE PROPOSED SYSTEM 

The effectiveness of the proposed system was evaluated by 10 undergraduate students, who do not have 
much knowledge and experience of circuit making, in an actual class at the Tokyo University of 
Agriculture and Technology (TUAT). Each student learned the comprehensive topics of a transistor 
amplifier according to the steps described in the following subsections. 

3.1 Preliminary Learning of a Transistor Circuit 

An NPN transistor consists of two N-type semiconductors separated by a P-type semiconductor, 
forming three terminals, namely, emitter, collector, and base. As shown in Figure 2(a), the currents to 
the base and collector are symbolized as I B and 7 C , respectively. The learner constructs the circuit 
shown in Figure 2(a) using an NPN transistor (2SC1815). By selecting the preferred mode of virtual 
measurement (VCM, RCM, or the mixed mode), the user can measure the Vce~Ic characteristics shown 
in Figure 2(b). The learner studies the following features of the transistors using the virtual measurements 
of the proposed system: 

• When a voltage V cc is applied between the collector and the emitter and a current I B flows to the 
base, an NPN transistor allows a current I c . 

• The amount of I c depends on the amount of I B . 




v a [V] 

(a) (b) 

Figure 2. Static characteristics of an amplifier with a transistor; (a) circuit diagram, and (b) Vcir^c characteristics. 


3.2 Learning of an Amplifier with a Transistor 

To design and evaluate the usability of the functionalities provided in the proposed e-Fearning system, 
individual students learn the comprehensive topics (described in Section 1) of the amplifier with the 
transistor (2SC1815) shown in Figure 3(a). Figure 3(b) indicates the components and currents necessary for 
designing the amplifier. To acquire comprehensive knowledge on the working of the amplifier, individual 
students were required to complete the topics ( 1 )— (6), where a part of the manual to design the circuit, 
the characteristics to find the operating point of the amplifier, and a part of the web-based input form for 
evaluating the designed circuit are shown in Figure 4(a)-(c), respectively: 



Figure 3. Small signal-amplifier with a transistor: (a) circuit diagram, and (b) DC equivalent circuit. 
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(1) Let the collector current I c and the current amplification degree h FE be 5 mA and 125, respectively. 
Using the manual and the datasheet, calculate the necessary currents and determine the operating point 
P. 


(2) Upload the designed information to the remote education system using the web-based input form and 
check the correctness of the designed circuit based on the evaluation scheme included in the proposed 
system. 

(3) Construct the circuit using the preferred mode (VCM, RCM, or the mixed mode). 

(4) If the system indicates wrong parts on the image of the constructed circuit, make the necessary 
changes. 

(5) Using the function of virtual measurements, plot the input and output voltage signals and the frequency 
characteristics of the constructed circuit. 

(6) Using the function of virtual measurements, measure the voltage amplification ratio, gain, and 
bandwidth of the constructed amplifier and evaluate them using the system. 


3. AC Load Line 



Ic - Icp = -(Bc//Rl)(Vcx - Vcep ) 
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Figure 4. (a) Operating point and load lines that are indicated on Vcbt^c characteristics, (b) manual to calculate the 
circuit parameters, and (c) input forms to submit the information of the designed circuit. 


4. RESULTS AND DISCUSSION 

Ten undergraduate students at TUAT evaluated the proposed system through the experiments described in 
Section 3. Figures 5(a) and (b) show the virtual circuit constructed using VCM and the physical circuit 
constructed using RCM, respectively. The remote education system of the proposed e-Learning system 
provided the SPICE information for constructed circuits that were obtained from the circuit translation 
technique. As shown in Figures 5(c) and (d), individual students learned the working of the circuit designed 
by them, in terms of the output signal and frequency characteristics, through the virtual measurement 
technique. Moreover, the system allowed the experimenter to construct the large-scale circuit (multistage 
amplifier) using the mixed mode of the system. Figures 5(e) and (f) show the constructed multistage 
amplifier and the working of the amplifier, respectively. Individual students submitted their opinions to the 
remote education system. The following positive responses, validating the usefulness and effectiveness of 
the proposed system, were obtained from all students: 

• The web-based learning of circuit theories and analysis of transistors were effective because the 
complicated calculations pertaining to the circuit design were instructed explicitly. 

• The comprehensive studies of transistor circuits using the e-Learning system were instructive 
because the system allowed learning of both theories and experiments. 
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• The choice of three modes of circuit construction (VCM, RCM, or the mixed mode) provided by 
the e-Learning system enables users to select a preferred mode depending on the environment. 

However, the following technical suggestions were suggested for the improvement of the system: 

• The usefulness of this e-Learning system would be much greater if the circuit translation technique 
is also made applicable to the circuit constructed on a printed circuit board. 

• Improvements are expected in the e-Learning system for learning digital circuits. 



Figure 5. (a) Virtual circuit constructed using VCM and (b) physical circuit constructed using RCM. (c) Results of the 
virtual measurements of the constructed circuit showing the input and output signals and (d) frequency characteristics, (e) 

Constructed multistage amplifier and (f) its working. 


5. CONCLUSION 

This paper proposes a novel e-Learning system that aids in the learning of electronic circuits with transistors. 
The proposed system enables users to understand the comprehensive theories and experiments pertaining to 
electronic circuits. The usefulness and effectiveness of the proposed system were verified by 10 
undergraduate students, who served as experimenters on the system, in an actual university class. Positive 
responses, which pertain to the effectiveness and efficiency of the proposed system, were obtained from all 
the students. The technical suggestions from the users (Section 4) are necessary for improving the usefulness 
of the proposed system. 
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TECHNOLOGY, GENDER ATTITUDE, AND SOFTWARE, 
AMONG MIDDLE SCHOOL MATH INSTRUCTORS 
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ABSTRACT 

Technology has gained a firm stronghold in society as well as modern classroom. Students are assumed to have a natural 
aptitude for computers. Over the past decades, educational websites have appeared to be “interactive” and “to make 
learning fun”. This study employed quantitative method of research using 8 th grade instructors from South East Dallas 
Texas school district. Data was collected via Microsoft Excel and SPSS computation. The survey instrument tested for 
internal consistency and reliability. The seven Likert scale items tested nine variables for Pearson correlation. A t-test 
detected non-significance at the p > .05 level of probability indicating that male teachers seldom use technology to teach 
math than their female colleagues. A second t-test showed a significant difference at p < .05 level for difference between 
younger math teachers (male and female) versus their older counterparts. 


KEYWORDS 

Technology Attitudes, Teacher Attitude, Teacher Technology Use, Technology Integration 


1. INTRODUCTION 

The pervasiveness of technology in society has highlighted the need for schools to prepare students to take 
advantage of emergent technology tools (Keane, J. (2002). For this to occur, (Otero & Peressini, 2005) 
opined that “today’s classroom teachers must be prepared to provide technology-supported learning 
opportunities for their students” They also add that “being prepared to use technology and knowing how that 
technology can support student learning must be integral skills in every teachers’ professional repertoire”. 

Technology has changed the role of teachers from “masters” to “facilitators” for students in the learning 
process. As facilitators, teachers introduce tasks, answer questions, prompt discussions, and 
summarize/present outcomes (Yank, 2004). 

Middle school teachers have differing views on the use of technology in their classrooms. For instance, 
54% of teachers see technology tool as a communication tool; 40% say technology is for productivity; 34% 
posits that technology is for research purpose while only 23% say it is for problem solving (ISTE, 2007). 

Jaradat & Hoagland, (2009) notice three important advantages of using technology in mathematics. First, 
dynamic media (video, audio and multimedia) offers better learning and knowledge opportunities than static 
media by giving learners opportunities for dynamic interactions and inquiries with different components of 
the product. Second, new representational infrastructure offers opportunities to reintegrate previously 
achieved knowledge. Third, new systems of knowledge might be designed by employing infrastructure based 
on technology. 


2. GENDER ATTITUDE 

Attitude is everything. Teachers’ attitude (male or female) towards technology makes the difference. This 
assumption confirms the commonly accepted belief that attitudes affect behavior (Albirini, 2004). Existing 
variables such as access to equipment, administrative support, and time to mention just a few have strong 
effect on the use of technology (Demetriadis, et al., (2003), in a longitudinal study, (5 years) reported that 
teachers’ use of laptop, confidence level and skills improved remarkably but computer use with students in 
the classroom remained relatively low. Thus knowledge and skill acquisition depends on teachers’ attitude 
and their readiness to use technology (Koszalka, 2001). 
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3. GENDER AND TECHNOLOGY 

Many studies have been carried out to investigate the question of gender and technology Bame, Dugger & 
Sharp, (2005). A commonly cited reason for many teachers limited use of technology is lack of skills. 
Batane, (2004), posits that both new and veteran teachers feel inadequately prepared to use computers in their 
classrooms. Most researchers in the early studies found that attitude towards technology differed significantly 
between males and females, with males indicating greater knowledge and interests (Liao, 1999; Meelissen, 
2008; and Young, 1999). 

Other researchers like Boxer, Palmer & Daudgerty, 1998; Krendi & Broihier, (1990) found that females 
perceived technology as more difficult and less interesting than males did. According to Linn, (2002), this 
could be as a result of dominant culture of males in technological fields. Other possible reasons, for gender 
differences could be violence in computer games that appeal to the male population (Sharp, 2005). 

Recent studies however show that gender differences are no longer different. According to the American 
Association of University Women /educational Foundation, females perform as much as males if not better 
except that they just see them differently than males (Bain & Rice, 2006). 


4. THE PURPOSE OF THIS STUDY 

1. To explore the effectiveness of technology as perceived by male and female math teachers 
compared to the traditional approach that most teachers favor in South-East Dallas Texas. 

2. To explore teachers’ attitude (young versus old) towards the use of math technology (software) 

4.1 Two Research Hypotheses Were Formulated 

1. Male math teachers are less satisfied in the use of math technology (software) than female teachers. 

2. Younger math teachers are more open to the use of math technology (software ) 

4.2 Scope 

This study was targeted to a select section of a school district located in South East Dallas Texas. A sample 
from ten middle schools, drawn across the district was used. 

4.3 Methodology 

Only a select section of schools located in South East Dallas Texas district was used for this study. This 
includes a population sample of 21 Math teachers drawn from the 10 sampled middle schools. All of the 
teachers took part in the survey and that represented 100% participation. 

Out of this number, (21), the males teachers were nine (9) and their female counterparts twelve (12) 
representing 43% and 57% respectively. The participants’ age are in two groups. 39 and under was 
considered “young” while 40 plus was taken as “old”. The decision to have two main groups became 
imperative after the response showed the age bracket (18-22) isolated leaving two age brackets (30-39) and 
(40 and above) standing strong. 

4.4 Scales 

The survey instrument held fifteen questions; including demographics, multiple choice, and Likert scale 
items. SPSS 18.0 was used to analyze the data. The seven Likert scale items tested for Pearson correlation 
(table 1). 
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Table 1. Pearson correlations on seven variables using the Likert scale items. 



evaluation 

accessibility 

recommendation 

assessment 

mastery 

integration 

performance 

Evaluation 

Pearson 

Correlation 

1 

.102 

.203 

.581" 

.234 

.384 

-.084 


Sig. (2-tailed) 


.661 

.379 

.006 

.308 

.085 

.717 


N 

21 

21 

21 

21 

21 

21 

21 

accessibility 

Pearson 

Correlation 

.102 

1 

.110 

.142 

.412 

.452* 

-.253 


Sig. (2-tailed) 

.661 


.635 

.540 

.064 

.040 

.268 


N 

21 

21 

21 

21 

21 

21 

21 

recommendation 

Pearson 

Correlation 

.203 

.110 

1 

.188 

.161 

.316 

.042 


Sig. (2-tailed) 

.379 

.635 


.414 

.487 

.163 

.857 


N 

21 

21 

21 

21 

21 

21 

21 

assessment 

Pearson 

Correlation 

.581" 

.142 

.188 

1 

.257 

.|476* 

.199 


Sig. (2-tailed) 

.006 

.540 

.414 


.261 

.029 

.387 


N 

21 

21 

21 

21 

21 

21 

21 

Mastery 

Pearson 

Correlation 

.234 

,41| 

.161 

.257 

1 

.377 

-.088 


Sig. (2-tailed) 

.308 

.064 

.487 

.261 


.092 

.704 


N 

21 

21 

21 

21 

21 

21 

21 

integration 

Pearson 

Correlation 

.384 

.452* 

Hi 

.476* 

.377 

1 

.319 


Sig. (2-tailed) 

.085 

.040 

.163 

.029 

.092 


.159 


N 

21 

21 

21 

21 

21 

21 

21 

performance 

Pearson 

Correlation 

-.084 

-.253 

.042 

.199 

-.088 

.319 

1 


Sig. (2-tailed) 

.717 

.268 

.857 

.387 

.704 

.159 



N 

21 

21 

21 

21 

21 

21 

21 


**. Correlation is significant at the 0.01 level (2-tailed). 
*. Correlation is significant at the 0.05 level (2-tailed). 


The strongest correlations were evaluation and assessment at 0.58; integration and assessment at 0.48 and 
integration and accessibility at 0.45. 

Male (gender code 1) math teachers are less satisfied than female (gender code 2) math teachers with the 
use of math technology - a t-test was run. There was no significant difference between male and female 
teachers in this population. The achieved value for equal variance assumed -1.682 - does not reach up to the 
table look-up value given for rejection of the null hypothesis (1.729). Therefore the null hypothesis cannot be 
rejected. The hypothesis that male teachers are less satisfied in their use of technology to teach math does not 
hold in this sample. 


5. CONCLUSION 

In this study, there is a distinct difference between math teachers that are young (39 and under) versus math 
teachers that are old (40 and above). Male (gender code 1) are less satisfied than female (gender code 2), but 
there was no significant difference between gender. Secondly, a 2.637 exceeded the value where the null 
hypothesis significant tests (NHSTs) can be rejected. Further, there is no significant difference between male 
and female math teachers in the use of technology software. The achieved value for equal variance assumed - 
1.682 does not meet the table look up value given for rejection of the null hypothesis (1.729). Therefore the 
null hypothesis was not rejected. 

A follow-up study may perhaps be a comparative study on the effectiveness of technology in teaching 
math among male and female students. 
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ABSTRACT 

A comprehensive adjunct faculty training program is described, whose aim is to improve student perceptions of courses 
and programs in a private, not-for-profit MBA and Law degree granting university in San Francisco, The program is 
somewhat novel in that it uses (a) student input from open-ended responses on course evaluations to determine faculty to 
be addressed, (b) it uses in-place online learning platforms and processes to deliver training, and (c) it includes a plan for 
monitoring faculty improvement, through hopefully improved course evaluations, mentoring, and course championing by 
full time faculty members. 
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1. INTRODUCTION 

Faculty training and development remains difficult and diffuse for some higher education institutions. The 
many inherent boundaries (between academic schools, departments, and programs), coupled with resource- 
scarce school environments, often relegate faculty development to one-off or inconsistently presented events, 
with little accountability on the part of both organizers and attendees. Yet the higher education community 
understands full well that improved teaching results in a higher quality educational experience for students, 
and that perceived higher quality teaching and learning can result in improved word-of-mouth “marketing” 
for programs, as students and alumni discuss their educational experiences among themselves in in their 
communities of influence. This is evidenced in the current dialogues on assessment tools as indicators of 
course and program quality (Aston & Antonio, 2012). 

In the author’s institution, these dynamics have long been acknowledged. Until now, there were not 
resources available to address the teaching quality situation in a positive way. Recently, a position titled 
“Director of Teaching Excellence” has been created in our private not-for-profit university, and a faculty 
member with interest and background in pedagogy is charged with creating a visible and sustainable faculty 
training program. The program integrates institutional resources into an ongoing, reproducible set of 
teaching/ learning interventions that address negative student perceptions and concerns about their course and 
instructor experiences with our university. In short, the premise is that a primary, actionable set of assessment 
products are the actual words of the students themselves, discussing their particular course experiences, 
utilized as indicators in the structuring of faculty training. 


2. THE RATIONALE FOR THE PROGRAM 

The institution has much in-house and regional, state and national data about student perceptions of their 
educational experiences. Most if not all of those data are reported statistically, meaning that we nave 
numerical interpretations of our students’ responses. What these data invariably do not have are actual 
student comments. Thus, we end up with data featuring little context and no interpretation. Additionally, as 
part of our accreditation (Western Association of Schools and Colleges; see http://www.acswasc.org/), we are 
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on a consistent path to provide program reviews based on various standard and traditional assessment tools 
and processes. While there is much dialogue on the various support activities faculty need for teaching 
improvement, (What Types of Support Do Adjuncts Need? 2012) there appears to be less discussion of the 
role that actual student qualitative data, that is the actual written comments of students on course evaluations, 
can provide as guidance for teaching interventions that can improve student perceptions of their educational 
experiences 


3. THE DESIGN OF THE PROGRAM 

To address that situation, and to provide a solid grounding for the training program, actual student comments 
from open-ended questions provided on course evaluations are gathered, analyzed and categorized, and used 
to shape teaching interventions. This somewhat novel approach allows for targeted adjunct teacher training 
activities. It is understood that all faculty may benefit from training, but given that a large majority of our 
courses are taught by adjunct faculty, that group is thought to be the logical starting place for our program. 


4. COMPONENTS OF THE PROGRAM 

Institutional resources : one sometimes overlooked dynamic in designing teaching/learning interventions is 
existing institutional resources. It may be thought that “we don’t have the resources” to design and deliver 
workable faculty training and development activities. It’s a precept of this paper, and the teaching excellence 
approach, that institutions may have sufficient resources, considering in house systems and the easy 
availability of web-based tools. Our use of existing resources is described in other parts of the paper. 

Our university features a mature education-delivered-online entity named Cybercampus 
(https://elearning.ggu.edu/) through which we deliver 22 fully accredited degrees in Management, 
Information Technology, HR, Tax and Accounting as well as various ad hoc courses and programs. The site 
is also used as a portal for various institutional, school and department resources. This entity is used as a 
portal for information about the training program discussed here, as well as an archive for materials. It also 
supports group and individual email contact, and has a WIKI-like structure, which provides for easy group 
input and communication. 

Course evaluations; using course evaluations to identity courses and instructors in need of assistance: All 
courses offer a formal student evaluation opportunity once during the semester, to be administered in the 
latter part of the semester. These evaluations are given in person in face to face courses, and administered 
online in the online learning environment. Scaled questions about students’ experiences with the course are 
asked: course administration, course management, as well as questions specific to the instructor and his/her 
presentation of the course. Various administrative questions are asked also, that have no application to the 
training program. 

In addition to the scaled responses, open ended questions are asked. It is here that rich data are often 
found, but until the program under discussion was put in place, these comments represented only anecdotal 
data. Over the years, there has been, though, a clear sense that those written comments have represented a 
form of truth about our courses and instructors that remained untapped. 

The Process: It can be said that the process outlined below follows a systems model, (input = course 
evaluation written comments; process = analysis of those comments; output = training based on data) with 
feedback being provided by an instructor’s performance in subsequent courses. 

Overall course evaluation scores (5 point scale) for a control semester were ranked and placed into 
quartiles. Evaluations falling into the top and bottom quartile were chosen for analysis. The rationale is that 
we need to understand satisfaction factors among both high-performing and under-performing instructors in 
order to create appropriate training activities. Results were also divided between face to face and online 
courses, so as to make evident similarities and differences in responses between the two instructional modes. 

Overall, student evaluation scores in our university are high, with an average of 4.3 on a 5 point scale. An 
important irony is that this remains generally (not always) true, even when written comments about a course 
are negative. 
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Findings: students’ comments for high-scoring teachers and courses are, not surprisingly, the obverse of 
comments for lower scoring teachers and courses. In high-scoring course/instructor evaluations, students feel 
that positive factors are 
Teacher engagement 
Teacher enthusiasm and curiosity 
Effective course management 

Effective use of outside resources in a course: guest speakers, use of media (TedTalks, YouTube for 
example) 

Content Expertise (in all modes and courses, content expertise is not often mentioned) 

There is a direct connection between course content and the student’s professional life. (This type of 
response is important, given that our institution features “practitioner -based instruction” that is applicable in 
professional business environments). 

In low-scoring course/instructor evaluations, students report 

Performance expectations are not clear 

Course is managed poorly 

Lack of teacher engagement 

Lack of preparation 

Lack of respect for students 

Course uses dated materials; materials not updated from a previous semester 

Lack of understanding of the diverse classroom (we have many international students as well as our 
“target” student, the working business professional). 

Laculty are arrogant, egotistical and/or disrespectful 
Gradebook is not up to date (most often reported in online courses) 

Laculty are unprepared to teach in the online environment (online courses) 

Laculty are not present in the course; faculty enter activities rarely, or not at all (online courses) 


5. UTILIZING THE FINDINGS 

The findings are being used to design training interventions, to occur prior to a teaching semester, to provide 
the instructor time to make suggested adjustments. 

Within this time frame, a session is held for low-scoring adjunct faculty instructors, in face to face and 
online (Adobe Connect) modes. The sessions are intended to be archived. Instructors are highly encouraged 
to attend, both “as-needed” instructors and mentor faculty. Given that adjunct faculty are hired at will, 
improved course evaluation scores are seen as an attractive outcome. 

The outcomes from the previous recent evaluations will be used to direct content for the training events. 
In initial sessions, importance is placed on the design of the syllabus, as a course management document, as 
an organizing principle for the instructor and students, and, perhaps most importantly, as a visible contract 
containing implicit and explicit course management and performance expectations for both the instructor and 
students. 

In addition to design of the syllabus, faculty are acquainted with the results of the student comments, and 
various activities are presented intended to address those specific concerns. 

What has not been done yet is to present separate trainings for adjunct faculty teaching online courses. 
Student comments re online courses fall into distinct categories, but we have not yet been able to undertake 
training specific to online courses. 

It should be said that our online learning entity has highly skilled course designers who are active in the 
design of online courses, and who engage faculty throughout the design process as to effective online 
teaching. However, it is clear that some teachers who teach poorly online are requiring the additional impetus 
of training presented by peers in the teaching community. 
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As to tracking and subsequent interventions, we are at the beginning stages of our next steps. We will 
monitor the evaluation scores and student comments from courses the next time a “trained” faculty member 
teaches a course. This will be somewhat difficult, given that courses are not offered consistently across 
semesters. Another mitigating factor is that faculty may receive “lower quartile” scores when teaching one 
course, but not when teaching another. If an instructor continues to receive low evaluation scores, further 
intervention may be required, via other trainings, mentoring, through interactions with Course Champions, 
and perhaps by not being rehired. 


6. SUMMARY OF THE PROCESS; CONCLUSION 

A characteristic of this program that separates it from ad-hoc training is the use of actual student comments 
as input to shape training activities. 

We intend that this program results in an ongoing, reproducible faculty training process that will over 
time become enculturated, that will move among the other entities in the university, and that will survive 
personnel changes. Much of the management of faculty data can easily be managed by support staff by slight 
adjustments to formal and ad-hoc enrollment and faculty reports, easily generated by university data systems, 
leaving the implementation of training to the Director of Teaching excellence and mentors as well as Course 
Champions. 

The technology issues have proven relatively easy to manage. We use a combination of face to face / 
Adobe Connect session, and the sessions are archived. The online learning “portal” for the training program 
is easily accessible by all faculty in our School of Business. 

The content of the trainings can, we feel, remain generally the same over time. It is our experience that 
student comments about the perceived high and low points of their courses are consistent. Faculty who are 
successful teachers exhibited the same characteristics in the past as the do now, and probably will in the 
future. Likewise, weak points in teaching have been the same in the past as they are now. 

While several aspects of this program may be innovative, it is the use of course evaluation data over time, 
to both identify needs and track progress, that differentiates the program from others. 
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ABSTRACT 

Problem-based learning (PBL) challenges traditional views of teaching and learning as the learner determines, to a large 
extent with support from a skilled facilitator, what topics will be explored, to what depth and which processes will be 
used. This paper presents the implementation of problem-based learning methodologies in an online Bachelor’s program 
in adult education and digital technology. The processes of PBL in the online environment and the subsequent roles of 
learners, teaching assistants and instructors in the co-construction of the learning environment are examined. Implications 
for the institution and other research underway are discussed. 
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1. THE BA AEDT PROGRAM CONTEXT 
1.1 BA AEDT Program Description 

The Bachelor of Arts (BA) in Adult Education and Digital Technology (AEDT) program recognizes the 
growing importance of formal and informal lifelong learning as seen in the expanding professional 
development needs of post-secondary instructors. Similar requirements are apparent in corporate, 
community-based, professional and public sector workplaces. As the instructional demands of these large 
institutions and organizations continue to grow outside the traditional university and college -based education 
system, and as human resources departments turn their attention to the economic benefits of just-in-time 
online learning opportunities, there is an increased need for individuals with the knowledge, competencies 
and skills associated with teaching and learning in adult education environments in the digital age. The 
AEDT program intends to prepare a new kind of expert who is essential to the knowledge-based economy 
because they possess a broad social understanding of adult education and a specialized knowledge and 
competency-base of the use of digital technologies for learning. Current students in the AEDT program 
represent early - mid career adult learners who are working full time in higher education, healthcare, finance, 
government and manufacturing sectors. 

The AEDT program is designed for advanced entry by those with a post-secondary diploma. A typical 36 
hour (3 credit) course in the AEDT program is articulated in 12 weekly modules that include: (1) a collection 
of curated online resources comprised of a combination of 3 - 5 video clips of 5 - 10 minutes in duration and 
associated online readings which outline the contexts and/or situations within which problems can be 
identified; (2) online synchronous tutorials in Adobe Connect that are 60 minutes in duration. These are 
moderated by a Teaching Assistant (TA) and/or the course instructor and draw upon the analysis and 
synthesis questions posed in the video clip as the starting point for discussion; (3) online discussions in 
Blackboard or other asynchronous tools; and, (4) independent and group work on problem based learning 
(PBL) tasks as defined and negotiated by their collaborative teams. The 2014-2015 academic year represents 
Year Two for intake into the AEDT program. There are currently approximately 50 full time AEDT students 
with over 100+ students from other university faculties choosing to take one or more AEDT courses as 
electives. 
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1.2 AEDT Program Structure 

In keeping with the need for access to information age courses and programs, problem based learning (PBL) 
is central to the design of the AEDT program and is defined as “a curriculum model designed around real life 
problems that are ill structured, open ended or ambiguous... PBL engages students in intriguing, real and 
relevant intellectual inquiry and allows them to learn from these life situations” (Fogarty, 1997, p.2). In each 
of the courses in the AEDT program, students or learners are working in teams on a variety of problem 
scenarios or contexts that combine to make up a unit or section of the course. Learners are expected to work 
collaboratively to initially identify or create a problem as presented in the situations or contexts and then 
subsequently to propose solutions to the problem using any and all synchronous and asynchronous tools 
available (Savin-Baden, 2007). 

As further described in the AEDT Course Development Model (vanOostveen, 2013a), the PBL 
orientation of the program requires the setting of a context in which problems can be identified for students to 
investigate as part of the course work. Typically 2-4 PBL modules varying in length from 1 session to 4 or 5 
weeks are incorporated into each course (Savin-Baden, 2007). The activities, assignment and assessments in 
the course then become the vehicle for the creation of solutions to the problems. Assessment tasks are 
authentic and focused on process rather than content (vanOostveen, 2013b). Each course in the AEDT 
program has used one or more aspects of several of the five models of problem based learning outlined by 
Savin-Baden, (2000) as part of its design philosophy. In addition, the following PBL design principles 
(Engel, 1991) inform every course in the program: 1) active learning; 2) integrated learning; 3) cumulative 
learning; 4) consistency in learning and, 5) learning for understanding. 

The AEDT program design philosophy is informed by the Technology Competency and Use (TCU) 
framework which builds from a Community of Inquiry (COi) model (Garrison, Anderson and Archer, 2000) 
and “considers that a technology object serves as an interface between the user and: 1) other users, 2) stored 
information and 3) information processing tools or software” (Desjardins, 2014, para. 1). The TCU 
framework identifies four orders of technological competencies (technical, social, informational and 
epistemological) which are examined through the TCU survey instrument. The TCU survey has been used 
“to collect data in a variety of higher education settings and work is progressing on the analysis to explore 
trends in use as well as the associated competency and skill development” (Desjardins, 2014, para. 1). 

1.3 AEDT Program Facilitators 

Program instructors are drawn from the faculty ranks and augmented with sessional staff. There are also 
numerous teaching assistants working in the AEDT courses, all of whom are current graduate students in the 
Faculty of Education. The online pedagogical structure that underpins the AEDT program postulates that 
students, instructors and teaching assistants should be viewed as learners in the learning community that is 
instituted, each with equal standing from a power perspective but with different goals and responsibilities. 
Some of these learners (instructors and teaching assistants) primarily function as facilitators, stepping in to 
scaffold and support the community. While each facilitator diverges in different areas of content knowledge, 
each is committed to high standards of professionalism and aiding to the growth and expansion of knowledge 
for AEDT program learning community. Facilitators are required to possess and employ characteristics such 
as: empathy, intuitive awareness, introspection, and analytical skills. These combined with a collaborative 
team mentality appear to foster an effective online PBL environment (Hmelo-Silver, 2004; Hmelo-Silver, 
Duncan, & Chinn, 2007; Savin-Baden, 2007). 

Facilitators who have personally experienced the processes of PBL generate integrity as they tend to be 
genuinely empathetic towards students and show high levels of affective support to nurture learners (De 
Grave, Dolmans, van der Vleuten, 1999), leading to the establishment of stronger relationships. Facilitators 
in the AEDT program who display a genuine interest in learning collaboratively as a team with the other 
learners, remove the power dynamic from their role and instead integrate into the community of learners as 
equals. Facilitators can then model and encourage introspection and analytical skills with others and foster 
engagement in the co-creation of knowledge. Facilitators essentially become cognitive apprentices by 
modeling reflection, questioning strategies and critical thinking (Collins, Brown, Newman, 1989). Effective 
online PBL facilitators must also display an intuitive awareness (Hmelo-Silver, Duncan, & Chinn, 2007) 
when guiding others in the construction and integration of knowledge (De Grave, Dolmans, van der Vleuten, 
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1999). Expert facilitators who exhibit an instinctive awareness of each learner’s zone of proximal 
development (ZDP) and observe learners’ non-verbal or subtle cues challenge learners in a non-aggressive 
manner (Vygotsky, 1978). Therefore facilitators who integrate themselves as a member in the community of 
learners build robust relationships that allow all learners involved to prosper in the PBLO environment. 

The AEDT students, as equal members in the online learning community, also play an important role in 
the facilitation processes of PBL. As co-constructers of knowledge, the students need to be motivated 
intrinsically, communicate their conjectures with openness to constructive criticism and show perseverance 
with the desire to learn outweighing the discomfort that accompanies PBL (Hmelo-Silver, Duncan, & Chinn, 
2007). As students move away from extrinsic motivation of grades, towards an actual desire to grow and 
expand their knowledge, this shift empowers learners to take responsibility and cognitively engage in their 
learning (Mayer, 2004). Once they have accepted their responsibility within the community they begin 
intellectual play with other members by communicating and defending their ideas, thoughts and conjectures 
and scaffolding other learners ideas through rebuttal and questioning (Puntambekar & Kolodner, 2006). In 
the AEDT program the process of experiencing PBL initially can accompany feelings of frustration for each 
member of the learning community as learners adjust to the new freedom afforded to them when the walls 
and constraints of traditional learning have been removed. Learners within the community (students, 
instructors and teaching assistants) need to embrace this cognitive dissonance in order to experience a 
critically reflective “aha” moment (Bencze, 2014). Habermas (1971) explains this reflective moment as the 
knowledge experienced that leads to a transformed consciousness. It is during this moment that the learner 
may ultimately emancipates their entrenched constructs of traditional learning while both recognizing and 
accepting the new liberties and boundless options possible. In the AEDT program it is usually after this “aha” 
moment when all learners in the community may arrive to a deeper understanding of the processes of PBL 
and the roles and responsibilities of each member in the community. 


2. PBL USAGE IN THE AEDT PROGRAM 
2.1 What is PBL? 

PBL tends to challenge traditional views of teaching and learning since PBL, in essence, takes control of 
learning processes out of the hands of the teacher/expert and places it squarely on the shoulders of the 
learner. The learner determines, to a large extent with support from a skilled facilitator, what topics will be 
explored, to what depth and which processes will be used. Working from a video based case study which 
presents a real life context (Logarty, 1997), learners create a problem or a set of problems which will become 
the basis of subsequent inquiries. PBL is organized around the contexts from which problems are drawn. The 
contexts then must be relevant and authentic in order to provide links that the learners use to make 
connections to their own experiences and interests (Hmelo-Silver, Duncan & Chinn, 2007). 

A wide variety of problem orientations are used within the AEDT program. ‘Given’ problems, using the 
phraseology of Watts (1991), proceed from tightly constrained contexts within which the goal or desired 
situation and the strategies to be used are provided to the learner. Given problems are viewed in similar ways 
to Savin-Baden’s (2000) Model I scenario. Learners operating in these scenarios are expected to apply 
predefined information to the differential between the current situation and the desired situation (Sd-Sq and 
in so doing produce a solution to the ‘problem’. The emphasis is on ‘how to’ rather than constructing new 
understandings or knowledge. Another way to conceptualize these scenarios would be to view them as 
‘projects’ with a focus on the application of concepts to the focussed issue or ‘problem’. At the other end of 
the continuum, constraint-free, ‘blue-sky’ type problems are also represented in a number of courses in the 
AEDT program. Watts (1991) talks about these as ‘own’ problems or problems in which neither the 
goal/desired situations nor the processes to be used to reach the desired situations are provided. These aspects 
are then left to the learners to determine and decide. Savin-Baden’s (2000) Model V scenarios are similar in 
that they are characterised by providing learners with occasions to develop their own learning autonomy 
including “multiple models of action, knowledge, reasoning and reflection, along with opportunities for the 
students to challenge, evaluate and interrogate them. Students will therefore examine the underlying 
structures and belief systems implicit within a discipline or profession itself, in to not only understand the 
disciplinary area but also its credence” (p. 133). Lor example, when issues such as the juxtaposition of 
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technological and sociological determinism when effecting tool choice and use is considered, students must 
be given opportunities to explore the implications for their own lives as well as for society at large. It is 
important for educators to be able to anticipate the beneficial and detrimental effects of a particular 
technology on students and for educators to determine how student groups will react to certain technological 
tools and affordances. 

2.2 Strategies used in the AEDT Program 

The PBL continuum manifests itself regardless of the type, complexity or features of the situations that are 
represented in the AEDT program. Contexts are presented in the form of publically (Creative Commons 
Attribution Licenced) available YouTube video clips. YouTube was chosen as the platform of choice due to 
its easy to use features, as well as its relative reach in that it seems to be ubiquitously accessible through an 
Internet connection from anywhere in the world. Each of the clips was purposefully created by instructional 
developers and in ways that are designed to be viewed by individual learners prior to coming together to 
discuss the created problems. Discussions between and among learners about the problems are conducted as 
tutorial sessions that are held in browser based virtual audio/video conferencing rooms. During these sessions 
learner work groups are initially established around similar problems. The learners then proceed to clarify the 
problem setting, gather the knowledge and resources that will be required to work towards creating solutions. 
The instructor and/or teaching assistant (TA) facilitates the collaborative discourse by interposing questions 
that are designed to serve as scaffolded supports enabling the learners to remain within their zone of proximal 
development (Hmelo-Silver, Duncan & Chinn 2007; Vygotsky 1978). 

A third venue for interaction within the AEDT programs is centred on tasks and assignments which the 
learners undertake. Typically learners collaborate on these tasks and assignments within small groups of 3 or 
4. The work itself is done using a vast array of mostly open access/source tools that are currently available. 
This is provides freedom of choice for adult learners rather than confining them to work specifically within 
the constraints of closed systems such as a learning management system (LMS). The only stipulation that is 
placed on the choice of tools is that it needs to be sharable with the rest of the class and the location of the 
work be known by providing the URL. The tools used include: Skype, Google Drive, Prezi, TodaysMeet, 
Blogger, etc. In most courses, learners will be asked to post regular metacognitive reflections and progress 
reports in some type of discussion board. 

Since each of the courses in the AEDT program may use different definitions or models of PBL, the types 
of activities in which learners are engaged will also vary. In some courses, learners will engage ‘in 
explorations and analyses of data’, activities that are reminiscent of scientific investigations or inquiry 
learning and in others they will identify problems and ‘consult various resources to solve them’ (Hmelo- 
Silver, Duncan & Chinn 2007, p. 100) using processes that are very much PBL oriented. Therefore, for the 
purposes of the AEDT program, it is assumed that PBL and Inquiry Learning (IL) are similar as both require 
learners to collaboratively participate in ‘sense making, developing evidence-based explanations, and 
communicating their ideas (Hmelo-Silver, Duncan & Chinn 2007, p. 100). Lrequently in IL, learners will be 
required to pose questions, collect data and then to test hypotheses or conjectures regarding answers to the 
questions. 


3. IMPLICATIONS 

3.1 Redefining HE Degree Pathways 

By definition the AEDT program is situated to take advantage of the current interest in leveraging the 
potential of digital technology affordances in adult education contexts. It does this by providing alternatives 
for employment opportunities outside the Kindergarten to Grade 12 education system and capitalizes on the 
recent success of pathway programs, allowing qualified students with a college diploma to receive a 
university degree, while simultaneously decreasing the time required for completion. The AEDT program 
affords careers in HR management and training in the industrial, commercial and various other sectors by 
offering adult educators opportunities for professional development and advancement in adult education and 
human resource development. 
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With the course content numerous opportunities are presented to investigate the juxtaposition of prevalent 
but competing concepts, such as technological and sociological determinism. In addition, PBL allows for 
cognitive transformations, such as those described above, as learners explore adult education and the 
influence of digital technologies. Many of these transformations may be seen through a variety of theoretical 
lenses, such as the Puentedura’s (2003) SAMR Model. 

3.2 Institutional Implications 

In addition to the individual transformations that occurred as a result of the implementation of PBL in online 
environments, a wide variety of key institutional requirements which are necessary to support this type of 
program emerged from the development of the AEDT program. Work done by vanOostveen, Partosoedarso 
& Robertson (2013) identified nine areas requiring attention at the institutional level in order for the AEDT 
program to be supported in a manner that could be considered comparable with competing programs. These 
include: (1) promoting effective communication about online programs within the institution; (2) determining 
a fee structure that respects online learners and the institutional needs; (3) enhancing online student life; (4) 
creating online orientation sessions; (5) creating online practice and learning communities; (6) providing 
access to formal administrative functions such as the registrar’s office and program administration; (7) 
promoting the program and recruitment of learners; (8) increasing faculty flexibility and capacity through 
transformed understandings of pedagogy and learning; and, (9) developing and implementing specific online 
course and program evaluation processes. For an institution that is working to further establish its online 
presence, the ability for the AEDT program to help to elucidate areas requiring further attention has been 
critical to both the success of the program and the expansion of online offerings at the institution. 

In addition, work is currently underway that builds on the initial experiences of the AEDT program and 
the TCU instrument itself, to examine the differential in what Watts (1991) would deem “given” and “own” 
and Savin-Baden (2007) would deem as “Level V PBL” performance assignments and personal profiles 
generated by the TCU instrument. The outcomes of this research will aid in the further refinement of the 
AEDT thesis exit courses and lay the preliminary groundwork for two larger research studies focused on the 
experience of program and elective students respectively in the AEDT online synchronous PBL learning 
experience. Taken together, these research projects and institutional implications have the potential to further 
the understanding of the intellectual, social and academic engagement of students in online synchronous 
courses and the supports required for them to be successful. 
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ABSTRACT 

This paper presents a new mobile micro-broadcast (or near-me) Location-Based Service designed to promote face-to-face 
communication among users located within a given geographical area such as a University campus. Because the 
communication services provided are time dependent, the application decides whom to contact based on the geographic 
distance between the active subscribers of the service and the sender. The paper also presents some preliminary results of 
prototype evaluation. 


KEYWORDS 

Mobile computing, location-based services, face-to-face promotion, prototype evaluation. 


1. INTRODUCTION 

The current increasing penetration of smartphones in our society is a visible and undeniable fact (Canalys, 
2012; Ismail & Razak, 2011). According to data revealed by the IDC European Mobile Phone Tracker in 
Marques (2012), smartphones market penetration may be related to the current capabilities of these “small 
computers”, often nicknamed “pocket computers” (Sanchez et al, 2011). Also according to Sanchez et al. 
(2011), this phenomenon is even more evident in our Higher Education campuses: “The students, due to their 
age, are the main users and purchasers”. This current horizontal increasing penetration of smartphones can 
also be related to the inclusion of multiple sensors 1 . In this sense, the Global Positioning System sensor 
(GPS) has become the most used sensor by current mobile services and applications, giving rise to a new 
category of services commonly called in the literature Location-Based Services (LBS) (Schiller & Voisard, 
2004). 

1.1 Location-Based Services 

Location-Based Services (LBS) were defined by Virrantaus et al. (2001) as “services accessible with mobile 
devices through the mobile network and utilizing the ability to make use of the location of the terminals”. 
From a more user-centered view, Zipf (2002) defines LBS as “services for mobile users that take the current 
position of the user into account when performing their task”. In 2004, Schiller & Voisard (2004) defined 
LBS as “services that integrate a mobile device’s location or position with other information so as to provide 
added value to a user”. From a more system-oriented view, such services were also defined in 2005, by the 
international OpenGeospatial Consortium, as services that: “deliver information about location to people who 
are using wireless, position-aware devices such as cell phones and PDAs. (OGC, 2005) According to Zipf et 
al. (2012), the LBS concept was early proposed in 2002 by Brimicombe as an “intersecting field of various 
technologies, namely Geographic Information Systems (GIS), Internet, and mobile networks/devices”. So, 


1 In http://www.riehIer.com/sensors-and-sensors/ by Asad-Uj-Jaman 
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basically, LBS are using the potential and the capabilities of modern mobile devices, positioning technologies 
and mobile Internet to deliver user value-added information or services based on the user’s location. 
Although LBS seem something completely new and recent, this is not quite true. According to Zipf & Jost 
(2012), LBS had its origins in 1995 (at the same time as the Internet began expanding at a global level) from 
a mandate issued by the Federal Communications Commission (FCC) requiring that “... wireless carriers 
should be able to locate 911 callers within 50m of their location.” (VanderMeer, 2002) Regarding 
Position/Location Data alone, its use started even sooner - in the 1970s. In the first decade, it was just used 
by the entity that created it, presently the owner of the Global Positioning System (GPS) - the U.S. 
Department of Defense (U.S. Government, 2013). Since 1980s, several were the industries that, at a 
worldwide level, have been accessing position data through the GPS. 

1.2 The Common LBS Infrastructure 

In general, LBS are not “standalone applications but rather services that require some sort of network 
connectivity” (Zipf & Jost, 2012). This is why any LBS is based on the following four key basic components, 
or individual elements (Ferraro & Aktihanoglu, 2011): Mobile Devices, Service and Application Providers, 
Communication Networks and Positioning Component or Service (Figure 1). 



Figure 1. Four key basic components of any LBS 

The ability to locate a mobile user or keep track of is one of the most important elements in all the LBS 
chain. Without this component, the mobile device cannot calculate the user location and, therefore, the LBS 
will be useless. According to Ferraro & Aktihanoglu (2011), “... it’s become more common to be able to 
determine location via an API (Application Programming Interface) or software component to at least fix an 
approximate location”. In fact, that has been the case with the major current mobile players - Apple with iOS 
and Google with Android. These mobile players have been providing freely access to their Software 
Development Kits (SDKs) to any mobile developer, that can now take advantage of the GPS or other location 
technology and, further more, in conjunction with realistic graphical maps. According to Cruz-Cunha & 
Moreira (2011), this reality leads to a fact that must be taken into consideration: “... these new mobile 
devices have become a platform with many possibilities to develop research and implement new kind of 
LBS”. 


1.3 Some Examples of Location-Based Services 

Based on the existing commercial offer in the online Apple App-Store and among research projects in the 
area, the WIZI SMS LOCATION App and the CONNECTOR (Almeida et al., 2012) Research Project were 
identified as the most relevant mobile applications, considering the similarities with the goals of the present 
prototype: the promotion of face-to-face communication between users located within a given geographical 
area. The analysis made can be seen in Table 1. These two LBS examples have contributed and provided the 
basis for the implementation of the prototype ”I’m on campus and . . as will be seen further on. 
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Table 1. An excerpt of a checklist of the features/characteristics available on apps similar to “I’m on campus and . . . 

(August 2013) 


Features/Characteristics 

WIZI SMS LOCATION 

CONNECTOR 

Require registration to work? 

NO 

YES 

Send the current location by 

SMS or email 


Map provided by 

Google 


Context-sensitive help to the composition of the messages? 

Just one field-level assistance. 

NO 

micro-broadcast Time Dependent (near- me notification)? 

NO 


Implemented scenarios: 

1 . Travel Status 

1 . Academic Events 


2. Leisure activities plan 

2. Status 

“Follow Me” background mode (User tracking) 

YES 

YES 

On- App Chat: 

NO 

Instant Messaging (IM) 


1.4 Mobile Developer Platforms Trends 

As far as the development of native apps is concerned, according to data revealed by Mobile Vision (2014), 
both Android and iOS captured “over 94% of smartphone sales in Q4 2013”. This data may explain why 
these two platforms continue to be the prioritized choices for the current mobile developers: 84% of 

mobile developers are now developing for Android or iOS (or both), the two clear winners in the developer 
mindshare race”. This is because, at this point, for many developers the question is not which mobile 
platform to develop, but rather, which mobile platform to prioritize. 

1.5 Face-to-face Promotion 

To end this first section, related to this research study’s context and motivation, one more concept must me 
addressed. Maybe it can be translated by the following question, suggested by Grossman (2011): “Is face-to- 
face learning still important ”? Though, nowadays, there are several technological solutions to overcome the 
barriers of time and space, when one speaks of Education, several research studies show that face-to-face 
communication is still the preferred medium over all forms of computer-mediated communication (Daniel 
Johnson & Sutton, 2000; Hardesty, 2010). As Escotet (2013) contends, maybe because “Education as a 
general concept” should be neither confused with just “Instruction”, nor considered only “teaching or 
instruction, but social and cultural learning”. Therefore, as stated by Grossman (2011), nothing can be 
compared to face-to-face communication: “communicating face-to-face sends a message before you say a 
word. 

Having outlined the main technologies which were used to prototype a new micro-broadcasting LBS to 
promote face-to-face learning, as well as having contextualized and presented the main motivations behind 
this research study, in the next section, the prototype implementation will be described, covering these three 
main topics: Goals and Target Audience, System’s Model and Architecture, and Functional Aspects. Finally, 
in the last section, usability and preliminary user experience evaluation results are presented and discussed. 


2. PROTOTYPE AND EVALUATION METHODS 

The “I’m on campus and ...” mobile app prototype was developed in Objective-C for mobile iOS platforms 
and it uses Parse Cloud as the mobile Backend as a Services (mBaaS). 

2.1 The Prototype 

2 . 1.1 Goals and Target Audience 

“I’m on Campus and ...” implements a new kind of LBS that aims to provide some specific communication 
service support for people interacting in a limited geographic area, such as a Higher Education campus, to 
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promote face-to-face communication. Consequently, the main target audience will be teachers and students 
belonging to particular academic campus. 

2 . 1.2 System 's Model and Architecture 

Like any LBS, as it was stated before and illustrated in Figure 1, the “I’m on campus and ...” follows a client- 
server architecture comprising the “standalone application” (the iOS client mobile app) and “some sort of 
network connectivity ” (Zipf & lost, 2012) to be able to deliver the whole package of functionalities proposed: 

(1) The Mobile Device - any iOS mobile device (iPhone, iPod Touch or iPad) where the app must be 
downloaded and installed from the App-Store. (2) The Communication Networks - the existence of any 
network connectivity supported by the iOS device (3G/4G or Wi-Fi). (3) The Positioning System - the A-GPS 
is the location embedded engine/sensor present in the iPhone or iPad 3G/4G. (4) The Service and Application 
Provider - Parse was the mBaaS chosen. For the present research purposes, the Basic Free Plan is being used. 

2 . 1.3 Functional Aspects 

The communication services provided by “I’m on Campus and ...” are context-based and both context- and 
time-dependent, and include the possibility to offer or request specific support from active subscribers 
located within a specific geographic range. One of the key features of this App is a context-sensitive help to 
the composition of the messages. Therefore, the App offers to the user field -by-field assistance. Currently, 
the options available include the following scenarios: (1) The possibility to express availability for some 
specific activity to be performed at a specific location during a specific time frame; for example. Figure 2 
shows a professor notifying near-localized students of his/her availability to provide some academic support. 

(2) The possibility to notify other users of specific interest; for instance, this may be used for a student to let 
his/her classmates know that he/she is interested in getting in contact with colleagues keen to play chess in 
the next 3 hours and that they can meet at the cafeteria. (3) The possibility for a professor to notify his/her 
students or colleagues that he/she will be very busy during the next hour and would appreciate not to be 
disturbed within that time frame. 



Choose one of three predefined 
types of request: 



Composing a Contextual Request type . 

Make a Phone Call to: 

My Family emergency number 
My Best Friend emergency number 



Choose one of this academic 
request or customize your own: 


to share > 

to give > 

to exchange > 

to answer questions > 



So. remember the choices made so far 


(a) I’m available 


(b) to clear doubts . . . 




©01h:00m 

mws. ~"o~ ops £To 


(e) Complete 
message 


Figure 2. Field-by-field context-sensitive aids to compose a “I’m available . . . ’’-type message: I’m available to clear 

doubts about iOS programming, for about 01h:00m 


2.2 The Evaluation Methods 


The prototype evaluation (“I’m on Campus and . . . ”) is being carried out in two different stages. (1) In the 
first stage, related to the evaluation of Usability and User experience aspects, tests were carried out in a 
controlled environment, a lab and several offices, according to the availability of the participants. An option 
for a qualitative study was based on the idea that, as Nielsen Norman Group states “... you don’t have to 
measure usability to improve it”; “it’s enough to test with a handful of users and revise the design in the 
direction indicated by a qualitative analysis of their behavior” (Nielsen Normal Group, 2006). 

(2) In the second stage, a User experience field trial is being conducted at the Viana do Castelo 
Polytechnic Institute (IPVC) Campus. The focus of this study is the usefulness of the service and the 
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consolidation of the preliminary User Experience results obtained in the first stage. The methodology 
approach is also being a qualitative study. 

According to Nielsen Norman Group (2000), with a sample of 5 users we can achieve 85% of Usability 
Problems. This is why we invited two groups of 4 users each - one formed by 4 students and another 
composed by 4 teachers, to a total of 8 users for the first stage. The option by these two kinds of profile were 
based on the fact that they are the main target audience of the service provided by the App, with aims to 
promote the face-to-face interaction between these two major groups in a Higher Education campus. The 
instruments to collect the data in this evaluation phase were the following: (1) A survey questionnaire to be 
filled out by the participants during the Usability tests. (2) Three iOS Devices, two with the “Display 
Recorder” app to record on video the entire user interface interactions and, simultaneously, the audio from 
verbalized thoughts, doubts and comments, according to the ‘Think Aloud Protocol’. (3) An observation 
Checklist to be filled out by the researcher during each session to assure a rigorous data registration. The 
evaluation tests were performed with each participant individually and always in the presence of the 
researcher. 


3. USABILITY AND UX FIRST RESULTS 


The data collected in the first section of the survey questionnaire has the main goal of revealing the most 
relevant usability problems. It was filled out by a sample of 8 users and where each user performed the 
following 5 practical scenarios: (1) User sign-up, user log-in and user localization; (2) I’m on campus and I 
am driving to Aveiro in about 30 minutes and have space for 2 passengers with baggage; (3) I’m on campus 
and 1 need a ride to Porto for 1 person with baggage, in about 2 hours. (4) I’m on campus and I’m available 
to clear doubts about iOS, for about 3 hours; (5) I’m on campus and I’m interested in forming a study group 
on iOS programming, for about 1 hour. The second section of the same survey questionnaire, filled out at the 
end of each usability session, has the main goal of catching the preliminaries evidences about the User 
experience. 

From the first section, the data collected provides some evidence about (1) Task efficacy - all the tasks 
were successfully completed as requested; (2) Interface validation - although some usability issues were 
detected, in terms of ’’Easy to Use”, we think they are not so critical that justify a new version of the app to 
move to field trial (real usage scenarios tests). 

Table 2. Usability and User experience (UX) data collected from a sample of 8 users: value 1 corresponding to “I 

strongly disagree” and value 5 “I strongly agree” 


Aj - Affirmation i with i E [1-15] 

1 

2 

3 

4 

5 

X 

a 

A], It was simple to use the app. 

0 - 0 . 0 % 

0 - 0 . 0 % 

1 - 12.5% 

2 - 25.% 

5 - 62.5% 

4.5 

0.76 

A 2 . 1 managed to achieve the desired goals with the app. 

0 - 0 . 0 % 

0 - 0 . 0 % 

0 - 0 . 0 % 

1 - 12.5% 

7 - 87.5% 

4.9 

0.35 

A 3 . 1 used the app effectively. 

0 - 0 . 0 % 

0 - 0 . 0 % 

0 - 0 . 0 % 

2 - 25.0% 

6 - 75.0% 

4.8 

0.46 

A 4. 1 think that I need to know more about the app to use it in an efficient manner. 

1 - 12.5% 

1 - 12.5% 

0 - 0 . 0 % 

4 - 50.0% 

2 - 25.0% 

3.6 

1.41 

A 5 . It was easy to learn using the app. 

0 - 0 . 0 % 

0 - 0 . 0 % 

0 - 0 . 0 % 

3 - 37.5% 

5 - 62.5% 

4.6 

0.52 

Aft. The app does not need more help messages. 

1 - 12.5% 

2 - 25.0% 

0 - 0 . 0 % 

3 - 37.5% 

2 - 25.0% 

3.4 

1.51 

A 7 . The app. provided me the proper feedback to my actions. 

0 - 0 . 0 % 

1 - 12.5% 

1 - 12.5% 

1 - 12.5% 

5 - 62.5% 

4.3 

1.16 

A s . I felt that when I make an error, the app allows me to easily and quickly recover from it. 

0 - 0 . 0 % 

1 - 12.5% 

2 - 25% 

0 - 0 . 0 % 

5 - 62.5% 

4.4 

1.06 

A 9 . The app provided me all the information that I needed. 

0 - 0 . 0 % 

1 - 12.5% 

0 - 0 . 0 % 

3 - 37.5% 

4 - 50.0% 

4.3 

1.04 

Am. The information in this present app is clear and organized. 

0 - 0 . 0 % 

0 - 0 . 0 % 

2 - 25% 

2 - 25% 

4 - 50.0% 

4.3 

0.89 

A| |. The app has basically the functionality that I expect from an App of this category. 

0 - 0 . 0 % 

0 - 0 . 0 % 

0 - 0 . 0 % 

1 - 12.5% 

7 - 87.5% 

4.9 

0.35 

Ap. The app allowed me to explore functionalities by trial and error. 

0 - 0 . 0 % 

0 - 0 . 0 % 

1 - 12.5% 

1 - 12.5% 

6 - 75.0% 

4.6 

0.74 

Ap. Overall, I’m very satisfied with the app. 

0 - 0 . 0 % 

0 - 0 . 0 % 

0 - 0 . 0 % 

2 - 25% 

6 - 75.0% 

4.8 

0.46 

A 14 . I’ll use this app in the future. 

0 - 0 . 0 % 

0 - 0 . 0 % 

0 - 0 . 0 % 

3 - 37.5% 

5 - 62.5% 

4.6 

0.52 

A 15 . 1 consider this app very useful and I’ll recommend it to my friends. 

0 - 0 . 0 % 

0 - 0 . 0 % 

0 - 0 . 0 % 

1 - 12.5% 

7 - 87.5% 

4.9 

0.35 


From the second section of the survey questionnaire, the data was summarized in Table 2 and provides 
evidence about (1) Learning Time and Efficiency of Use (“Easy of Use”) - related to Learnability, 
Effectiveness and Efficiency quality metrics - the majority of users declared that the interface interaction was 
very simple to use, very easy to learn and very efficient; (2) Memorability and User Errors - related to 
Help and Feedback, Error Recovery and Information Over-load quality metrics - some users declared that 
they need some more help and feedback messages; (3) Subjective satisfaction - related to Satisfaction, 
Future Interest and Usefulness quality metrics - all users confirmed the usefulness of this new micro- 
broadcast LBS in Campus scenarios and also declared their future interest in using the app. 
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4. CONCLUSION 

Despite the relevance of these evaluation activities, it is important to underline that these tests were made in a 
controlled environment - a laboratory. So, a field trial was required in order to provide further evidence 
about: (1) What are the most frequently used scenarios and under what circumstances the app is mostly used? 
(2) With which perception each user stayed in terms of qualities, gains and weaknesses of the app? (3) 
Finally, with which perception each user stayed in terms of control, privacy and safety when using the app? 
Although these tests are still underway in ESTG/IPVC campus, we are confident that with the 
accomplishment of these field trials we will understand more in-depth the real interest in having this kind of 
new mobile micro-broadcast, or near-me, Location-Based Services in a campus. More precisely, with real 
usage scenarios tests we are very convicted that we can achieve a better understanding of how much this new 
mobile service will develop and facilitate the interrelationship and interaction between users located within a 
given geographical area, and how useful it will be to have a service of this kind on a campus to promote face- 
to-face learning. 
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ABSTRACT 

This paper describes the utilisation of a technology-facilitated professional learning network (PLN) for pre-service 
teachers, centred on chemical demonstrations. The network provided direct experiences designed to extend their 
pedagogical content knowledge on demonstrations in Chemistry teaching. It provided scaffolded opportunities to 
collaborate as they ‘negotiated their identity’ as chemistry educators. Using technology to facilitate conversations around 
the use of chemical demonstrations was seen as advantageous as it provided an explorative workspace to discuss and 
refine pedagogical approaches. Importantly, it was anticipated that through these personal experiences the pre-service 
teachers would better understand ways to utilise technology within a PLN. The design of the PLN, and two examples of 
discussions designed to phenomenologically explore the teaching and learning associated with chemical demonstrations, 
ably facilitated by technology, are described here. 


KEYWORDS 

Professional learning network. Science, pre-service education. 


1. INTRODUCTION 

Today’s pre-service teachers are active users of a wide range of social technologies. Whilst their perceptions 
on how to utilize technology for learning varies, for the most part they share an obligation to use technology 
effectively (Linehan & McCarthy, 2000, Redman & Rodrigues, 2008). Within the Science Education 
program at the Melbourne Graduate School of Education, The University of Melbourne, the pre-service 
teachers are provided with strategically placed science education learning opportunities by explicitly 
embedding collaborative, engaging and integrated eLearning experiences within teaching practice. Research 
evidence (Redman, 2013) indicates that situating these new technologies within cognitively challenging, 
collaborative and engaging contexts leads to development of positive identities for pre-service teachers as 
learners and teachers with new technologies. 

This paper presents one aspect of ongoing research to document and identify the thinking and pedagogical 
practices of pre-service teachers as they utilize technologies as learning and reflective tools. The researchers 
pay close attention to the pre-service teacher’s perceptions of technology for effective Science learning, as 
well as their visions for technology use for their Science teaching practice. This research is being supported 
by an eLearning Learning and Teaching Initiative (LTI) grant (The University of Melbourne). As pre-service 
teacher educators, this research also informs our own practice, and helps align Science teaching in the 
subjects with students’ current usage of technology (as well as helping us support their future needs). 

This research reports on the utilization of a technology-facilitated professional learning network (PLN) 
with Chemistry pre-service teachers. Explicitly, the PLN provides direct experiences that extended their 
pedagogical content knowledge on chemistry demonstrations in Chemistry teaching practice. Implicitly, 
opportunities exist to collaborate as they ‘negotiate their identity’ as chemistry educators, and to develop an 
identifiable and shared language for teaching and learning, focussing on discursive practice, incorporating 
‘do’ and ‘say’ of the educator and the learners, is supported by the pedagogical standpoint that meaning is 
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co-constructed in discussion with others. A key premise is that meaning is further improved by making 
learning and the language associated visible to the learner (Hattie, 2003). Supporting choice, curiosity, 
discovery and collaboration are seen as vital to the 21 5,1 century learner (Hayes, 2006), as is placing language 
at the centre of empowered teaching and learning (Bourdieu, 1994). 

In Chemistry Education, which is largely the study and application of abstract concepts of atomic matter 
and sub-atomic interaction, educators and learners have always been challenged to teach and learn on 
multiple levels simultaneously. Students are required to think on a macroscopic level, that is the visible, the 
descriptive and functional, as well as the sub-microscopic level, that includes the invisible, but what explains 
why chemical substances behave as they do, and also the symbolic level, incorporating the visual, algebraic 
depictions of chemical understanding (Johnstone, 1982, 1993). Building an appreciation of pedagogical 
approaches that can be utilised to facilitate knowledge construction on these multiple levels (Gilbert et. al., 
2002) has become an important aspect of Chemistry pre-service teacher education. 

One such pedagogical approach to address the multiple levels is the chemical demonstration. The teacher 
demonstration, either with or without student involvement is a key teaching practice for Science pre-service 
teacher to learn to master. This involves questioning techniques, understanding methods of meaning making 
and facilitating the assimilation of concepts of Science through concrete observations. 

Using technology to facilitate a conversation around the use of chemical demonstrations was seen as 
advantageous as it provided an effective way to support the explorative workspace to discuss, improve and 
‘fine tune’ pedagogical approaches not only for individual demonstrations (timing, use of questioning, 
observations to highlight, opportunities for student involvement etc.), but also for the use of demonstrations 
as a whole. Importantly, it was hoped the pre-service teachers would begin to appreciate the benefit of these 
discussions, including a broad range of collaborative feedback and reflection experiences, occurring within a 
PLN, ably facilitated through technology. These approaches, along with a few examples of their use, are 
described below. 

It is notable that we tried to create a collaborative environment for learning that had invited the 
technology to show effective uses. Salomon, Perkins and Globerson (1991) stated that “it is not technology 
alone affecting minds, but the whole ‘cloud of correlated variables’, technology, activity, goal setting, 
teacher’s role, culture... exerting the combined attempt” (p. 8). 


2. METHODOLOGY 


2.1 Participants 

The participants were a cohort of Chemistry method pre-service teachers enrolled in their first semester of the 
Master of Teaching (Secondary) course at the University of Melbourne. During their first week, the pre- 
service teachers were provided with a link to an on-line survey, of which 50 of 56 enrolled students 
responded. The survey focused on a number of areas including; their use and relationship to new 
technologies; perceived benefits of using technology in the classroom; goals and vision for teaching and 
learning; perceptions of how ICT can be used to support their own professional learning. The survey also 
asked some chemistry-specific questions including about their own chemistry education background, their 
perceptions of effectiveness of chemistry demonstrations, and their current confidence in Science pedagogy. 

The survey responses are being coded and analysed using Positioning Theory (Harre, 1997) and 
referenced against Bloom’s taxonomy (Anderson & Krathwohl, 2001) to determine their existing perceptions 
and approaches for using technology. Comparing these to a similar survey completed at the end of the 
semester provides some insight as to how their perceptions of effectiveness of technology changed, for using 
technology for their professional learning, and their emerging confidence in teaching chemistry. 


370 



International Conference e-Leaming 2014 


2.2 Design of the Professional Learning Network 

Each week, two-three pre-service teachers present a chemistry demonstration to the rest of the pre-service 
teachers, around a central theme for the week (acid-base, gases, teaching junior levels etc.). As part of this 
presentation, they describe what they see as the challenges of teaching and learning associated with the theme 
for the week, and how their demonstration could be used and conducted as an educative experience. 

A series of collaborative feedback and reflective experiences then follow. Depending on the week, two- 
four fellow pre-service teachers are pre-allocated to participate in an online discussion around the particular 
demonstration presented that week. These online discussions were framed by the use of one of four different 
diagnostic tools that gather perceptions of effectiveness, and the perspectives of the teacher and learner. 
These are described in more detail in the next section. 

All discussions took place in google drive documents, in a shared google drive folder between the pre- 
service teachers and the educators. The pre-service teachers were given a session on how to utilize and 
collaborate within the google drive documents together as a group at the start of the semester, but other than 
that the pre-service teachers were left to decide on how best to participate and complete the collaborative 
exercises together. 

At the start of the next week, the entire cohort gather around two-three tables to have an informal 
‘summing up’ discussion. This includes the presenter of the demonstration, those who were allocated to 
participate in the online discussion, and other interested members of the cohort. Whilst largely an informal 
‘chat’, this was seen as vital to the researchers, as it brought to the fore the meta-cognitive nature of these 
discussions, and provided further opportunity for voice and to question, providing a more enriching 
personalized learning experience. 

Finally, the demonstration presenter is also tasked with writing a brief reflection and posting this in 
another shared exploratory workspace. Edmodo was used for this purpose, given that it provided a highly 
personalized experience, and allowed for the cohort to post and share resources with each other in a free, 
secure social learning platform. The posts written by the demonstration presenters summarized some of the 
feedback and comments from the online discussion and the follow-up discussion. These were written as a 
self-reflection on their ability to conduct the demonstration as an educative experience. 


3. RESULTS AND DISCUSSION 

Four different approaches were trialed as collaborative experiences facilitated by technology. Each were 
chosen for their ability to gather and document perspectives of the participants, from a phenomenological 
view, i.e. what the teacher and the students were ‘seeing’ and ‘doing’ during the delivery of the chemistry 
demonstration. Each required little or no modification to be utilized in an online exploratory workspace. 

The four approaches were a SWOT (Strengths, Weaknesses, Opportunities, Threats) analysis; the 
SCAMPER (Substitute, Combine, Adapt, Modify, Put to another use. Eliminate, Reverse) technique; a 
Collaborative Interactive Discussion (CID), and; a Personal Meaning -Making Map (PMMM). The first two 
were known to the pre-service teachers. For the later two, class-time in an early session was allocated to 
allow the teachers to undertake one of each, to promote how these could be used to make thinking visible, 
document and discuss their perspectives, and how this could be done in an online social environment. 

Figure 1 below shows a CID about a chemistry demonstration, lead iodide precipitation with different 
reagents in excess (Commons & Hoogendoorn, 1990), which took place in an online environment. Briefly, 
the pre-service teachers are all shown the same ‘focus statement’, and are then given different ‘focus 
questions’. Each participant fills in the response in the first row, with a 1-2 word response in the left column, 
and a longer response in the right. They then move on to responding to another participant’s initial response 
to a different focus question and so on (Ryan and Jones, 2014). 

An analysis of their utilization suggested that the participants were able to deeply explore the discursive 
practice of each chemical demonstration. Facilitated by technology, this gave individuals time to consider 
their response and the comments of others, and provided an equal voice to all participants. One negative was 
that as the duration of the discussion was considerably longer than if it had occurred during a class workshop, 
participants often had to wait for their turn to respond, which perhaps limited the buildup of narratives in the 
shared workspace. 
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24 

25 Focus statement 

26 I 

27 A good 'mole' demonstration makes an amount of substance visual to the learner. A great 'mole' demonstration 
goes further and shows the amount of substance to not be defined by its mass and/or its volume. 


29 


30 


Focus question 

31 

32 


What could be done to enhance the key observation in this demonstration. What do we want to learner to focus 
on, and make use of building new knowledge? 


It was very hard to actually see the total height of the solid/liquid mixture, as well as the position of 
the interface. Preparing the samples more carefully and earlier, to give a longer sedimentation time, 
could make things clearer. Using a thinner stirrer to mix the solutions together would help. 

33 


Slow 

though 

Maybe by inserting the second solution slowly into the bottom of the measuring cylinders might help 
"neaten" up the experiment so we can easily visualise the heights of the precipiates. 

34 


not clear 
interface 

Much bigger samples, in bigger vessels will allow clearer view of the solids-liquid interface and the 
main key of this demonstration will be much clearer 

35 


Yes, 
but. . . 

Perhaps another compound that precipitates so students could compare and do a few formulas at 
once may also enhance their experience and learning 

36 




37 




38 




39 





Figure 1. Excerpt from an online collaborative interactive discussion (CID) around a chemical demonstration. 

Figure 2 below shows a personal meaning-making map (PMMM) of one participant written after 
observing the Carbon pillar demonstration (Commons & Hoogendoorn, 1990), and the follow-up discussion 
had with another participant (which would appear underneath the map in the google drive document). Inside 
the online document, the pre-service teacher was easily able to gather their perspectives using the figure 
drawing function, as shown here, or take a photo of a hand-drawn picture and upload the picture with any 
device. Having generated the discursive ideas to discuss, a second participant (another pre -service teacher in 
the cohort) then came into the online workspace to ask clarifying questions, providing an opportunity for the 
map maker to elaborate on their ideas and further identify and document their meaning making processes 
(Falk & Dierking, 2002). 


Collaborator 2’s Drawing 



Discussion ( Collaborator 1 as educator, Collaborator 2 as 
learner) 

Why do you think the small beaker makes it work better? 

The column rises more., possibly because of the volume to 
surface area ratio of the beaker. 

What were you thinking about when you wrote "Is it 
burning?... no flame"? 

I was wondering why it turned black if there was no flame, it 
didn't look like it was burning. 

I think a student would ask that question too 
What would your answer be? 

I believe it's because all that is left in the pillar is carbon, 

which is black. 

hmmmm 

Do you think there is a way to relate this demo to a real world 
application? 

This demo shows that there is a lot of energy in sugar, even 
though it's not a form of energy that students are used to 
seeing. It may have applications in the food industry. 


Figure 2. Excerpt from an online PMMM drawing and discussion (modified from original) 

Each of the four approaches trialed, including the SWOT analysis and SCAMPER technique, did not 
appear to lose any authenticity or capacity for free choice by moving the discussion into an online 
exploratory workspace. As described above, the ‘drawn out’ nature of the discussion was seen as the main 
negative, but perhaps this also highlighted to the participants how learning with others in an online social 
environment can be a constant, ongoing journey of reflection and discovery. 

A follow-up survey of participants occurs at the end of semester, and is used to analyze and track how 
their perceptions of the use of ICT for professional learning have changed, and their ‘goals’ and ‘visions’ 
statements for their use of ICT in teaching practice will again be coded and analyzed to determine if the 21 bt 
century skills we as higher education educators have tried to demonstrate have started to translate into their 
own teaching practice. 
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4. CONCLUSION 

This study examined how a combination of pedagogical approaches were utilized to help redefine teacher 
identities. Through these experiences we anticipate these teachers are more likely to be willing to incorporate 
technology in their classrooms. A number of approaches were utilized to facilitate novel collaborative and 
reflective online discussions around language and discursive practice associated with a series of chemistry 
demonstrations. Technology ably facilitated this approach, and its affordances allowed for more voice and 
authentic responses from its participants. A survey of initial perceptions of the use of ICT for professional 
learning showed a broad range of opinions and abilities, and so it is expected the upcoming follow-up survey 
will show a cohort of pre-service teachers who are building an appreciation of how technology can be used to 
analyze and improve their professional learning and teaching practice. 


ACKNOWLEDGEMENT 

The authors acknowledge the grant support of the University of Melbourne for this project, and the support of 

the Science Education team at the Melbourne Graduate School of Education. 

REFERENCES 

Anderson, L. W., Krathwohl, D. R. (Eds.), 2001. A taxonomy for learning, teaching and assessing: A revision of Bloom's 
Taxonomy of educational objectives: Complete edition. Longman, New York. 

Bourdieu, P., 1994. Language and symbolic power. Polity Press, Oxford. 

Commons, C., Hogendoorn, B. (Eds), 1990. Demonstrations for Secondary School Chemistry, Chemistry Education 
Association, Melbourne. 

Falk, J.H., Dierking, L., 2002. Lessons without limit: how free-choice learning is transforming education. AltaMira Press, 
Walnut Creek, USA. 

Gilbert. K., De Jong, O., Justi, R., Treagust, D.F., Van Driel, J.H. (Eds.), 2002. Chemical Education: Towards Research- 
based Practice. Kluwer Academic Publishers, Dordrecht. 

Hattie, J, 2003. Teachers make a difference: What is the research evidence? Australian Council for Educational Research 
(ACER) Research Conference, 19-21 st Oct. Melbourne, Australia. 

Harre, R., 1997. Forward to Aristotle; the case for a hybrid ontology. Journal for the Theory of Social Behaviour, Vol. 
27, No. 2-3, pp. 173-191. 

Hayes, D., Mills, M., Christie, P., Lingard, B., 2006. Teachers and schools making a difference. Allen and Unwin, 
Sydney. 

Johnstone, A.H., 1982. Macro- and micro- chemistry, School Science Review, Vol. 64, No. 227, pp. 377-379. 

Johnstone, A.H., 1993. The development of chemistry teaching: A changing response to changing demand. Journal of 
Chemical Education, Vol. 70, No. 9, pp. 75-83. 

Linehan, C. and McCarthy, J., 2000. Positioning in practice: Understanding participation in the social world. Journal for 
the Theory of Social Behaviour, Vol. 30, No. 4, pp. 435 - 453 

Redman, C & Rodrigues S, 2008. Researching the relationships in the Technologies of Self: Habitus and Capacities. 
Australian Association of Research in Education (AARE) International Education Research Conference, Brisbane, 
Australia. 

Redman, C. (Ed.) 2013. Successfid Science Education Practices: Exploring what, why and how they worked. Nova 
Science Publishers, New York. 

Ryan, J. and Jones, M. (Eds), 2014. Effective Teacher Education: Partnerships, Reflective Practice and the Place of 
Technology, Sense Publishers, Rotterdam (in press) 

Salomon, G., Perkins, D., Globerson, T., 1991. Partners in cognition: Extending human intelligence with intelligent 
technologies. Educational Researcher, Vol. 20, No. 3, pp. 2-9. 


373 



ISBN: 978-989-8704-08-5 © 2014 


THE EFFICIENCY OF E-LEARNING ACTIVITIES IN 
TRAINING MENTOR TEACHERS 


Laura §erbanescu and Sorina Chircu 

Teacher Training & Social Sciences Department, Politehnica University of Bucharest 
Splaiul Independentei, Nr.313, Sector 6, Bucuresti 


ABSTRACT 

In the present article we aim to present the general framework related to the laws and practice of mentorship in Romania, 
as well as the results of a related investigation of the POSDRU project concerning the training of professional insertion 
mentors, “From start to success - national program for the professional insertion mentorship of teachers”. The paper aims 
to measure the degree of efficiency of the project’s e-leaming platform. In this sense we are using an evaluation model 
for e-learning activities called Kirkpatrick’s progressive evaluation model on 4 levels: Reaction, Learning, Behaviour, 
Results. To obtain a maximum congruence with these 4 progressive levels we use the following research methods: 
document analysis and survey-based research. The sample was comprised of 342 teachers from the lower secondary 
system, participants in the project in order to obtain an extra mentor qualification. The data processing was done in the 
IBM SPSS software package, the obtained results indicating a maximum level of efficiency of the project’s e-learning 
platform both in assuring a high level of learning as well as implementing information on a behavioural level and the 
impact on the organizational context through the effects it can produce. 


KEYWORDS 

E-learning platform, Kirkpatrick’s model, mentorship, teacher training 


1. INTRODUCTION 

In the present article we aim to present the general framework related to the laws and practice of mentorship 
in Romania, as well as the results of a related investigation of the POSDRU project concerning the training of 
professional insertion mentors, “From start to success - national program for the professional insertion 
mentorship of teachers”. In the national law of Education no. 1/2011, in article 236 (1), letter c, 
apprenticeship is seen as part of the initial training: every apprentice teacher undergoes a 1 year long 
practical stage in a school, under the coordination of a mentor teacher. During the apprenticeship period we 
can observe an ever growing exit from the system of the young apprentice teachers (over 500 annually), 
which reconvert to other fields of activity, not only for financial reasons, but also for the lack of support 
programs aimed at overcoming difficulties in adapting to the professional environment. Every year 1000 
graduates receive tenure. The weak methodical training is reflected in the results at the tenure contest: from 
7241 enrolled in 2007, 1764 (33%) were accepted; from 8890 enrolled in 2008, 2264(47,3%) were accepted. 
Although the legal framework for supporting an apprentice teacher is new in Romania, it’s important to 
mention that in the Romanian Education system theoretical and practical experiences exist, concerning the 
mentorship activity, several national and international projects of this type being organized in the last few 
years: the MOSS (Mentor Observation and Support Scheme) aimed to create national standards for the 
mentorship activity through elaborating a structure to support mentors (of all specialties) through the 
involvement of all mentorship partners; Romanian trainers have delivered such training activities, 
consultancy and experience exchange with countries such as Hungary, Ukraine, countries from the former 
Soviet Union, Latvia, Bulgaria (http://asmero.ro/index. php?module=cms&page=79); the START project - 
The purpose of the project is to create and implement a training model that leads to the systemic professional 
development of apprentice teachers in the first 2 years of their teaching career 
(http://www.euroed.ro/index.php/home). 
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The project “From start to success - national program for the professional insertion mentorship of 
teachers” held between February - December 2013 aimed to improve the professional insertion of apprentice 
teachers from the lower secondary school educational system through the development of a national, formal 
and coherent system of apprenticeship. The professional insertion concept outlines the integration process of 
a young apprentice or new employee in the professional environment for which he received specific training 
(L. Ezechil, 2013, page 1). The improvement of professional insertion of teachers involves that this process is 
so well done that it provides teachers who are performers to the education system starting from their career 
beginnings and on the other hand it retains them into the system. The teacher is not yet fully qualified, so he 
is called a candidate or trainee, even if he runs certain teaching activities. This form of training that belongs 
to the initial training period is present in numerous European countries: Germany, France, Luxembourg, 
Portugal, Austria, Scotland, Flolland, England, Cyprus, and Slovenia. There is, in education systems from 
other countries, a transition period of 2-3 years, in which the young teacher, the apprentice, does not have the 
status of a definitive teacher, but obtains it if he fulfils certain criteria concerning the quality of his teaching 
activity, promoting certain forms of examinations. This is the case in Greece, Spain, and Italy. Only half of 
the countries in Europe offer new teachers a certain method of support in their first year of teaching. In the 
school systems there are various guidelines for transferring initial training of teachers to their actual 
professional activity. For example, the duration of programs varies from 7 months to 2 years. In the majority 
of countries, mentor teachers, most of the time in cooperation with the school's director or other more 
experienced teachers, have the responsibility to help teachers start up. Experienced teachers, invested with 
the role of tutors (mentors, trainers, counsellors, coordinators) have the task to offer support to candidates 
from 3 points of view: 1. training for their teaching activity; 2. knowledge and adapting to the school 
environment; 3. monitoring and evaluation of professional activities of the candidates. Therefore, we can see 
the effort of education systems in several countries to facilitate the transition of future teachers towards their 
full professional activity (L. §erbanescu, 2011, page. 76-77). 

The project “From start to success - national program for the professional insertion mentorship of 
teachers” aimed to harness the experience of senior teachers therefore growing the attractiveness of the 
teaching profession and to create new education professions (professional insertion mentor, mentor trainer), 
the creation of a body of nationally certified mentors and the actual assistance of apprentice teachers from 
different specializations. One of the specific objectives of our project is to assure the accessibility, the 
efficiency, the transparency and the later sustainability of the project through the creation of the online 
platform. In order to reach this objective several activities were held: the design and implementation of the 
online platform with e-learning content and educational resources for training activities; the development of 
the software support for the online platform; the editing of the usage guide for the online platform; the 
submission of mentor training materials to the online platform; producing a project virtual library accessible 
to all teachers involved in the project; the editing and publishing of content in the e-learning sections; 
maintenance of the project’s web site. The training program pursued the development of several 
competences, including an ITC blended -learning training module aimed at using the online platform. The 
monitoring activities in the project comprised the assistance of apprentice teachers by the mentors in the 
project through online communication via the ITC platform. Constant communication happened on the online 
platform, the online monitoring including the progress evaluation of the apprenticeship program, allowing 
corrections as applicable. 


2. THE DESIGN OF RESEARCH 

The paper’s purpose is to measure the degree of efficiency of the project’s e-learning platform. Working 
hypothesis: “We expect that the e-learning platform of the - From start to success, national program for the 
professional insertion mentorship of teachers - project will represent a virtual learning environment useful 
for mentors, destined to develop their competences in order to facilitate the professional insertion of 
apprentice teachers”. To verify the working hypothesis we are using the Kirkpatrick model, a model that 
measures the efficiency of e-learning through a progressive evaluation of 4 levels: reaction, learning, 
behaviour and results. According to Kirkpatrick’s model, the first level - the reaction - measures what the 
course attendants believe about the e-learning platform in general; the second level - learning - measures 
what the attendants actually learnt. The third level - the behaviour - measures the degree of information 
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implementation, their transfer into behaviour. The fourth level, the results, measures the impact on the 
organizational framework through the effects it produces. To obtain a maximum level of congruence with 
these 4 levels we are using the following research methods: document analysis and survey-based research. 
The sample is comprised of 342 lower secondary school teachers who have more than 18 years experience 
and who participated in the project to be certified as mentors. The data processing was done in the IBM SPSS 
software package, using descriptive statistics by calculating frequencies. 


3. DATA PROCESSING AND INTERPRETATION 


3.1 Document Analysis 

Through the document analysis of projects we can observe the online activities delivered by experts and 
mentors of which we can name: creating homework, feedback for homework, quality assurance online 
procedures, online follow-up procedures, online ongoing evaluation activities, the support & monitoring of 
mentor’s online chat activities with apprentices (www.mentorat.cpi.ro, pe-mentorat.cpi.ro). To emphasize 
results we will make a short description of the activities. Course manuals in electronic format were created as 
well as extra resources useful for mentors (the mentor’s kit, the apprentice’s portfolio, quality assurance 
procedures, etc). The implementation of the professional insertion program comprised the theoretical 
framework preparation and the instruments (online and face-to-face surveys) through which a feedback 
regarding the qualitative level of the program was obtained. All the organizational details have been posted 
on the project’s platform thus facilitating a good dialogue between the management team, experts, mentors 
and apprentices. 

3.2 The Survey Based Research 

The items of the surveys were created to verify the way in which the 4 progressive levels (specific to 
Kirkpatrick’s model) were attained. Level 1, the Reaction - measures what the course attendants believe 
about the online platform in general. We verify the attainment degree of this level through the result analysis 
of the following item: “To what degree do you appreciate the project’s online platform components?”. 

Table 1. The degree of appreciation regarding the following levels of the e-learning platform 


Very large Large Moderate Small 

Frequency Percent Frequency Percent Frequency Percent Frequency Percent 


Site graphics 

141 

41.2 

132 

38.6 

62 

18.1 

7 

2.1 

Site usability 

205 

59.9 

117 

34.2 

19 

5.6 

1 

0.3 

Site sections 

211 

61.7 

107 

31.3 

22 

6.4 

2 

0.6 

The quantity of 
information 

205 

59.9 

116 

33.2 

20 

5.9 

1 

0.3 

The quality of 
information 

194 

56.7 

132 

38.6 

15 

4.4 

1 

0.3 

The timeliness of 
information 

192 

52.1 

123 

36.0 

24 

7.0 

3 

0.9 

The Forum 

205 

59.9 

114 

33.3 

21 

6.1 

2 

0.6 

The Chat 

201 

58.8 

122 

35.7 

17 

5.0 

2 

0.6 


The obtained data show (Table 1) a favourable reaction of attendants regarding the various components 
and aspects of the platform. Thus, we observe that they appreciate the platform especially because of its 
usability, 59,9% of attendants appreciate that they navigated very easily on the platform’s pages and 61,7% 
appreciate to a large degree the website’s sections. Of these we can observe very favourable reactions for the 
forum (59,9%) and the chat (58,8%). Level 2, Learning - measures what the attendants actually learnt. We 
verify the degree of attainment of this level through the result analysis of the following item: “To what extent 
the intellectual interaction of attendants with the learning material and with the online tasks was 
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encouraged?”. The various activities designed into the project and held on the platform prove their efficiency 
judging by the resulting values that indicate the appreciation of attendants regarding the fact that all activities 
encouraged their intellectual interaction with the learning materials and the online tasks, to a large and very 
large degree (Table 2). 

Table 2. The degree of intellectual interaction of apprentices with learning materials and online tasks 



Frequency 

Percent 

Valid Percent 

Valid Very large 

213 

62.3 

62.3 

Very small 

1 

0.3 

0.3 

Large 

110 

32.2 

32.2 

Moderate 

18 

5.3 

5.3 

Total 

342 

100.0 

100.0 


Concerning the online evaluation tasks of topic 5, Informational techniques in the mentorship activity and 
the measure they were learning and development tasks of ITC abilities, from Module 2, homework was 
evaluated from topics such as: a. making a Google Drive Form and submitting its link on the platform; b. 
making a Wiki glossary of useful links for the mentorship activity; c. making a SWOT analysis for an e- 
learning platform. 

Level 3 - Behaviour, measures the degree of information implementation and its transfer into behaviour. 
We verify the degree of attainment of this level through the result analysis of the following item: “Did you 
use in your mentorship activity the materials on the platform?”. 


Table 3. Usage of the platform’s materials in the mentorship activity 



Frequency 

Percent 

Valid Percent 

Valid Very large 

213 

62.3 

62.3 

Very small 

1 

.3 

.3 

Large 

110 

32.2 

32.2 

Moderate 

18 

5.3 

5.3 

Total 

342 

100.0 

100.0 


After the training program the certified mentors started their activity of mentoring apprentices. The 
platform also offered mentors a virtual library that had resources useful in the mentoring activities: the 
mentor’s guide, the mentor’s kit, the apprentice’s portfolio, the legal framework, etc. These resources, as well 
as the information obtained during the training period through the forum and the chat regarding the experts’ 
answers to the various problems that were foreseen in working with the apprentice, represent for the mentor a 
guiding mechanism when needed. The values (Table 3) indicate the fact that the mentors through 
implementation have capitalized the experience and information acquired dining the training period, the 
platform offering such training resource with a large degree (62,3%) of applicability in the daily school 
environment. Level 4 - Results, measures the impact on the organizational framework through the effects it 
produces. We are interested to see whether in the mentors’ opinion, the program contributes to the creation of 
a national system of assisting and supporting junior teachers in the lower secondary school system, online 
teaching activities on the ITC platform contributing to this coherent teaching system. The impact on the 
organizational level is a large one, the mentors considering to a degree of 99,4% (Table 4) that the program 
contributes to the creation of national system of assistance and support of junior teachers in the lower 
secondary school system, online teaching activities on the ITC platform contributing to this teaching system. 
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Table 4. Online teaching activities on the ITC platform create a coherent teaching system 




Frequency 

Percent 

Valid Percent 

Valid 

Yes 

340 

99.4 

99.4 


No 

2 

0.6 

0.6 


Total 

342 

100.0 

100.0 


Concerning the platform’s limits, identified at the level of possible difficulties met by mentors in using 
the platform we find a few worth mentioning: the difficulty of working simultaneously on several open pages 
in the platform; more suggestive keywords should be used on the platform. 


4. CONCLUSIONS 

Working hypothesis: “We expect that the e-learning platform of the - From start to success, national program 
for the professional insertion mentorship of teachers - project will represent a virtual learning environment 
useful for mentors, destined to develop their competences in order to facilitate the professional insertion of 
apprentice teachers” is confirmed. The reactions of the attendants to the online platform are favourable ones, 
59,9% from the mentors appreciate they navigated very easily on the platform’s pages and 61,7% 
appreciating to a very large degree the sections found on the website. Also worth mentioning are the requests 
of mentors posted on the online platform regarding the maintenance of the platform as long as possible, even 
after the end of the project, so that all mentors can have access to the discussion forum and the existing 
resources. The impact on an organizational level is a very large one through the effects it can produce on the 
Romanian educational system, the mentors considering in a proportion of 99,4% that the teaching activities 
on the ITC platform form a coherent teaching system and the program contributes to the creation of a 
national system of assistance and support of apprentices in the lower secondary school system. 
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ABSTRACT 

This paper presents the development process and framework used to construct a transportation app that uses situated 
learning, augmented reality, and communities of practice. Autism spectrum disorder (ASD) is a neurodevelopmental 
disorder that can cause social impairments as well as the limit the potential for the individual to achieve independence 
(American Psychiatric Association [APA], 2013). The use of technology to support learning in individuals with ASD has 
been well documented, however gaps exist in the pedagogical approaches that are being used. Future research 
opportunities into the pedagogical implications of this app are highlighted. 


KEYWORDS 

Situated learning, communities of practice, augmented reality, authentic learning, autism spectrum disorder, intrinsic 
learning 


1. INTRODUCTION 

The use of technology is well documented for the enhancement of learning, with countless pieces of 
software, apps, devices, and tools available to help educate individuals. Recently there has been an increase 
in the number of tools developed for individuals with special needs as more and more iPads, iPods, and 
similar devices are making their way into the classroom. Many of these apps were developed for individuals 
with autism spectrum disorder as tools to increase communication or comprehension abilities, however the 
majority of these tools are developed for use within the K-12 system. Unfortunately, there appears to be a 
lack of tools developed for adults with autism spectrum disorder (ASD), creating a gap in resource 
availability. Additionally, tools that have been developed often use behaviourism as their primary approach 
to learning, which can lead to a stimulus response pattern, as opposed to the creation of new understandings 
(von Glasersfeld, 1989a). 

This paper focuses on the development of an iOS transportation app that makes use of situated learning, 
augmented reality, and communities of practice. The app was developed and designed for high-functioning 
young adults and adults with autism spectrum disorder. The development of this app stems from a gap in 
available tools that facilitate authentic learning in individuals with autism spectrum disorder. The iOS 
application is currently being tested before use in a research setting; therefore this paper focuses on the 
development process of said application. The research question propelling this study is whether an app that 
makes use of situated learning, augmented reality, and communities of practice will increase intrinsic 
learning in high-functioning individuals with autism spectrum disorder. 
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2. CONTEXTUAL INFORMATION 

2.1 Autism Spectrum Disorder 

Autism spectrum disorder is a neurodevelopmental disorder that is typically manifested in childhood, and is 
sustained throughout the individual’s life (American Psychiatric Association [APA], 2013). Individuals with 
autism spectrum disorder have persistent social impairments, including social -emotional reciprocity, non- 
verbal communication, developing and maintaining relationships (APA, 2013). These individuals may also 
have restricted and repetitive patterns of behavior, including motor stereotypes, strict following of routines, 
and resistance to change (APA, 2013). Autism is a spectrum disorder; some individuals may have extreme 
social impairments while others may have mild social impairments (APA, 2013). Individuals can also range 
from having no/mild to severe intellectual impairments (APA, 2013). Because of the range of intellect within 
the population, the design of this app and related study focuses on individuals who are high-functioning, 
those who have the ability to take transit on a semi -dependent or independent basis as well as those who are 
have technical competencies to use an iOS device. Individuals with ASD often have low adaptive skills, 
making planning and organizing very difficult (APA, 2013). This, paired with a low tolerance for change, as 
well as a high prevalence for anxiety and depression in the young adult and adult populations, makes it 
difficult for adults with ASD to achieve independence (APA, 2013). 

One such area in which individuals may experience difficulties is using public transportation. Individuals 
with ASD who rely on transit, such as buses, streetcars, subways, and trains, may find navigating transit 
schedules particularly stressful or challenging. These individuals may employ the transit system to attend 
appointments, community functions, school, or work. As a consequence, they may have to rely on 
Community Support Workers, caregivers, guardians, or parents, which, in turn, impedes their sense of 
independence. This app has been developed to provide information to these individuals in a situated learning 
environment. 

2.2 Situated Learning 

Situated learning is used to form the framework for the design of the app; situated learning approaches 
learning as the construction of knowledge by the learner, rather than the transfer of information from one 
individual to another (Wehlage, Newmann, & Secada, 1996; von Glasersfeld, 1989b). Additionally, this 
knowledge is acquired actively by the learner and is gathered through their experiences within their authentic 
environment (von Glasersfeld, 1989b). When learners have an active role in their education, when they are 
responsible for what they learn, they are able to maintain a sense of ownership and control of their education 
(von Glasersfeld, 1989a). To achieve an authentic environment where learning can occur, it must be situated 
within a highly contextualized setting (Brown, Collins, & Duguid, 1989). The nature of these environments 
fosters learning as individuals are practicing in authentic situations with their own peculiar cultural practices 
(Brown, et ah, 1989). By engaging in this highly contextualized and authentic environment, learners are able 
to transfer knowledge to new situations and settings (Brown, et ah, 1989). 

This pedagogical approach was a foundational component of the framework that supports the design of 
the app. The app is intended to be used by the individual out in the field as opposed to a laboratory or 
classroom setting. This, according to Brown, et ah (1989), is to ensure that the individual is immersed within 
the same contextualized setting that they will be experiencing, thus promoting transfer of learning (Lave, 
1991). Learning within the same environment within which the app was intended to be used will be 
beneficial to individuals with ASD, increasing their potential to navigate transit systems independently. 

2.3 Augmented Reality 

In addition to providing an environment that is situated, the app needs to provide supports to the user in terms 
of navigation, location, and transit information. To do so, the app was built using augmented reality as a 
means of providing those supports. Augmented reality (AR) is defined as an environment that combines real 
and virtual information, is interactive in real time, and is registered in 3D (Azuma, 1997; Feiner, MacIntyre, 
& Seligmann, 1993). The use of AR supplements reality with information or tools that would not be available 
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in the physical environment (Azuma, 1997). Additionally, compared with virtual reality where users are 
immersed in a virtual world, AR allows the user to be present in the physical world (Azuma, 1997). Common 
examples of AR tools are Head Mounted Displays, such as Google Glass, Head Up Displays, which can be 
seen in military aircrafts and some automobiles, AR Apps and Games, which superimpose characters, 
graphics, text, or audio, into the physical world. Educational applications of AR can range from manipulating 
3D geometrical objects (Kauffmann & Schmalstieg, 2003) to providing virtual models of objects (Azuma, 
1997). 

In this case, AR is used to supplement the environment by providing visual tools as well as auditory 
information to the user. This app makes use of this by providing turn-by-turn navigation to the user, GPS 
information, as well as auditory warnings for upcoming transit stops. Through the development process, the 
app went through a number of changes that reduced the amount of augmented reality that could be used 
within the app. This is discussed with more details in section three. The most recent model of the app 
contains auditory augmentation, providing the user with real-time transit information, as well as visual 
augmentation, in the form of visual cues. 

2.4 Community of Practice 

As previously discussed, individuals with ASD may have social and communication impairments, which can 
interfere with their ability to build and maintain relationships, as well as learn from peers (APA, 2013). As 
adults these individuals may also have poor psychosocial functions, including independent living and gainful 
employment (APA, 2013). This substantiates the need for peer interaction support, as well as in the 
community at large, as individuals with ASD often rely on caregivers and guardians for participation in these 
social interactions. This app is designed to facilitate legitimate peripheral participation, as discussed by Lave 
and Wenger (1991). During this process, an individual becomes a part of the community of practice by 
participating in ongoing activities, making use of scaffolds, and will eventually be able to help others, 
transforming from a ‘newcomer’ to an ‘old-timer’ (Lave, 1991; Lave & Wegner, 1991). 

The app provides transitioning supports to the individual, as would an ‘old-timer’ within the community 
of practice. Lave and Wegner (1991) describe a community of practice as a domain where individuals share 
knowledge through active participation in the cultural practice. Due to the social impairments that individuals 
with ASD face, this app is designed to support these individuals to actively participate within the greater 
community. 


3. APP DEVELOPMENT 

The Transit App was built in conjunction with the Laculty of Education and the Laculty of Business and IT at 
the University of Ontario Institute of Technology. The app design was developed by a graduate student as a 
part of an ongoing thesis research project and was programed by three undergraduate students. Currently, the 
app is still in development, as it still needs to be tested before using it for the research project. The app has 
gone through many revisions over the development process, however the main purpose of the app is to 
provide supports to an individual with ASD as they navigate the Toronto Transit Commission (TTC), which 
is comprised of subways, light rail transit lines, streetcars, and buses. The basic structure of the app is to 
provide the user with transit schedule information, turn-by-turn navigation, as well as audio and visual 
indicators that inform the user that they are approaching the desired stop. To use the app, users will either 
type in their starting destination or use the current location as determined by their GPS signal. Lollowing this 
they will enter the desired location, or search through previously selected destinations in the app’s history. 
The app will then show a map of their desired path and when users select the ‘Go’ button, see Ligure 1, the 
app will provide directions for the user. As seen in Ligure 1, the directions provided indicate the time, 
location, route for the selected trip. These directions will update as the user arrives at each point, as well as 
make changes as necessary. These features were included to provide navigation support to increase the user’s 
autonomy, transfer of knowledge, and participation within their community. 


381 



ISBN: 978-989-8704-08-5 © 2014 



Figure 1 . Screenshots of Transit App 


Through the design process, a number of changes to the app had been made. Originally, it was desired 
that the app could provide opportunities for users to make amendments to the route, for example the user 
could tag a specific bus stop as always running late, or that the route typically has a nice driver. This was 
thought to enhance the community of practice by increasing participation of the user. However, this was not 
added due to server constraints. Additionally, original designs of the app had a timer that would indicate 
when the transit vehicle was set to arrive, however this would not be accurate due to a lack of GPS location 
of the specific vehicle. Although turn-by-turn navigation is still present in the current model of the app, it was 
originally planned to use the camera feature of the device to overlay these directions onto the physical world 
as depicted on the device’s screen. This feature was not included due to developer limitations. There had 
been a number of issues during the design process, including the availability of the programmers due to their 
status of full-time students. Server problems and computer malfunctions also caused a number of delays. 


4. CONCLUSION 

The aim of this paper is to highlight the design and development of an augmented reality app that uses 
situated learning and communities of practice to promote authentic learning in young adults and adults with 
autism spectrum disorder. This app promotes learning as knowledge construction through a social 
constructivist perspective (von Glasersfeld, 1989a). The design and pedagogy of this app provide practical 
applications that extend to other developmental and intellectual disorders, which provide future areas of 
research. The research applications of this app will be tested through an ethnographic study with the 
participation of young adults and adults with autism spectrum disorder as they navigate between locations 
using TTC and navigation directions provided by the app. 

Currently, the app is still in the developmental stages with testing still occurring. The process of 
designing and developing an app has proven to be a longer experience for all students involved than initially 
thought with many changes and revisions occurring based on technological complications. However, this 
process has provided countless learning opportunities for these students, providing a large area for growth 
and development. 
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ABSTRACT 

Information technology is now ubiquitous in higher education institutions worldwide. More than 85% of American 
universities use e-learning systems to supplement traditional classroom activities while some have started offering 
Massive Online Open Courses (MOOCs), which are completely online. An obvious benefit of these online tools is their 
ability to automatically grade exercises submitted by students and provide immediate feedback. Most of these systems, 
however, provide binary (“Correct/Incorrect”) feedback to students. While such feedback is useful, some students may 
need additional guidance in order to successfully overcome obstacles to understanding. We propose using a Case-Based 
Reasoning (CBR) approach to improve the quality of feedback Computer Science students receive on their programming 
exercises. CBR is a machine learning technique that attempts to solve problems based on previous experiences (cases). 
The basic idea is that every time the instructor provides feedback to a student on a particular exercise, the information is 
stored in a database system as a past case. When student experience similar problems in the future, knowledge contained 
in past cases is used to guide the students to a solution. While the system will provide detailed feedback automatically, 
this feedback will have been previously crafted by human instructors, leveraging their pedagogical expertise. We describe 
a system of this kind, which is currently under development, and we report results from a preliminary experiment. 


KEYWORDS 

Case-based reasoning, e-leaming, e-assessment, immediate feedback. 


1. INTRODUCTION 

Higher education institutions around the world are rapidly adopting information technology to help students. 
According to Chen et al. (2010), more than 85% of universities in America make use of Learning 
Management Systems (LMSs), such as Moodle and Sakai. Such systems allow students access to course 
materials, discussion forums, and other resources. LMSs facilitate the online submission of assignments or 
quizzes, in addition to managing enrollments and grades. Researchers have found that using LMSs to 
supplement the traditional classroom environment has been beneficial to students (Chen et al, 2010). 

Some institutions have recently started providing Massive Online Open Courses (MOOCs), where the 
entire learning experience is online. Organizations like EdX and Coursera offer online courses from leading 
universities. All instruction is done using pre-recorded video lectures, and communication with instructors is 
facilitated through online forums. Quizzes and assignments are submitted online and are graded either by 
fellow students (peer grading) or automatically (e-assessment), with feedback often provided instantly. 

Immediate feedback has been shown to facilitate deeper learning (Epstein et al, 2002) and is generally 
regarded as an advantage. Most automated grading systems, however, only provide binary feedback, in the 
form of “Correct/lncorrect". In the case of an incorrect solution, these automated graders are not able to 
provide guidance to the student. Some intelligent tutoring systems attempt to provide richer feedback, but 
this approach is not trusted by many educators (Van Lehn, 201 1). 
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In this paper, we propose a methodology for improving the quality of feedback provided by automated 
grading systems. The proposed framework is built upon the principles of Case-Based Reasoning (CBR). 
While the idea of using CBR for e-learning applications is applicable to a wide range of academic disciplines, 
we focus on improving the quality of feedback given to undergraduate Computer Science students on 
solutions to programming exercises. 

The rest of the paper is organized as follows: Section 2 introduces the reader to the Case-Based 
Reasoning methodology and provides examples of systems where CBR has been successfully applied. 
Section 3 provides the details on our proposed e-learning system. Section 4 presents the results of 
preliminary experiments, and Section 5 contains a discussion and concluding remarks. 


2. BACKGROUND AND RELATED WORK 

Case-based reasoning is a machine learning technique that uses knowledge from past experiences (or cases) 
in order to find solutions to current problems. Since case-based reasoning is a lazy technique, it builds up its 
knowledge base by simply storing past cases into a database. A case is defined as a description of a problem 
along with a solution to the problem. For example, the technical support department of an Internet service 
provider may employ a CBR system. A typical case in their database would have a description of the 
symptoms, say “Modem LED blinking rapidly”, and a solution, for example, “Try restarting the modem”. 

A new problem being encountered is called a “query”, presenting a description of the problem symptoms. 
The CBR system uses its knowledge base to try to find a solution to the query. There are typically four steps 
in the process. 

1 . Retrieve: In this step, the CBR system finds all cases whose problem descriptions are similar to 
the query. The most critical component here is the metric used to measure the similarity between 
the query and the past cases. 

2. Reuse: Using the solutionis) retrieved in the previous step, construct a solution to the query. 

3. Revise: Evaluate the quality of the solution provided in the last step. 

4. Retain: Based on the outcome of the revise step, decide whether to store the query and the 
solution as a case in the database, or not. 



Figure 1. The case-based reasoning methodology, adapted from Lopez (2013) 

Figure 1 illustrates the steps of the CBR process. For more details on the methodology, see Lopez (2013). 
CBR systems have been successfully applied in customer support centers, as shown in Allen (1994), while Jo 
et al. (1997) used a CBR approach to predict bankruptcy in Korea. Begum et al. (2010) surveyed a number of 
CBR systems used in the health sciences and found that these systems are being used for a wide variety of 
tasks, such as diagnosis, treatment planning, and training of medical personnel. 

CBR has also been applied to the educational sector, although it has not received as much attention as it 
has in other domains. Jonassen and Hernandez-Serrano (2002) proposed a CBR system to support problem 
solving using stories, and Ballera et al. (2013) proposed a CBR system to personalize the e-learning 
experience of students by sequencing the topics being presented. Wiratunga et al. (2011) presented 
RubricAce, a CBR system designed to assist instructors who use rubrics for grading students’ work. 
Rubric Ace suggests feedback comments to instructors once they have assigned grades according to a rubric. 
Instructors then decide how to use the suggested feedback to provide summative evaluations to students. One 
of our goals is to provide high quality immediate formative feedback, to the degree that such is possible. To 
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our knowledge there have been no previous attempts to use CBR in order to improve the quality of 
immediate feedback provided to students by e-learning systems. In the next section we describe a framework 
for achieving this. 


3. PROPOSED SYSTEM 

We now present the details of our proposed methodology for improving the quality of feedback generated by 
automated grading systems. We assume that a system for providing “Correct/Incorrect” feedback already 
exists. In the Computer Science domain, it is easy to implement a system that will distinguish correct from 
incorrect programming assignment submissions. Upon creating a programming exercise, the instructor 
provides test cases that are used to verify the correctness of student-submitted solutions. This is the “Grading 
System” module in Figure 2. When a student submits his/her first solution to a given exercise, the code is 
sent to the grading system and if it is determined to be correct, the student is given positive feedback. If, 
however, the solution is incorrect, the CBR component of the system comes into play. 

First, the system searches the case base for past cases that are similarly incorrect to the current 
submission. Two programs are similar with respect to incorrectness when they both contain the same bugs, or 
suffer from the same problems. A possible metric for establishing incorrectness similarity is to treat two 
submissions as similar if they both fail on a common set of test cases. This metric was used in the 
preliminary experiment described in Section 4. Other possible metrics, such as static code analysis, are be 
briefly discussed in section 5 and will be further explored in future studies. 

The past cases that are retrieved will contain pedagogically appropriate guidance on how to resolve the 
problems that are being experienced. Pedagogical appropriateness is determined by instructors who generate 
the feedback. They may generate different kinds of feedback for different students, experiencing the same 
problem. The system can use student-specific as well as submission-specific features to determine when a 
given kind of feedback is to be generated. 

Before the feedback is presented to the student, the system determines whether human intervention is 
necessary, which would be when the system is unable to retrieve similar cases, or when the student has 
repeatedly failed to rectify a particular problem, indicating that the feedback received has not been helpful. If 
it is determined that intervention is needed, a human instructor will manually examine the feedback and 
update it as necessary before it is given to the student. If intervention is deemed unnecessary, the system- 
generated feedback is passed directly to the student, who uses it to update and resubmit his/her solution. 

The resubmitted solution is graded again. If it is correct, it shows that the feedback that was generated by 
the system was, indeed, helpful to the student, and the entire submission history of this interaction with the 
student is used to update the case base. If the resubmitted solution is still incorrect, it is pushed through the 
CBR process repeatedly, including potential human intervention by members of the teaching team, until the 
student succeeds at the exercise. 



Figure 2. A flowchart of the proposed system 
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4. PRELIMINARY RESULTS 

At our institution we use an online grading system to automatically evaluate programming assignments 
completed by undergraduate students on a weekly basis. The correctness of each student submission is 
determined by compiling and running their code against predefined test cases. We have collected submission 
data from several undergraduate computer science courses over the past 2 years. 

As a proof-of-concept we have implemented a simple CBR system in order to gauge the potential impact 
of the methodology. We use a simple incorrectness similarity measure, where two programs are declared 
similarly incorrect if they fail on a common subset of the test cases. 

For the purposes of this experiment, we clustered submissions based on the incorrectness similarity 
measure above in order to determine whether instructor-generated feedback for one member of a cluster will 
be applicable to the rest of the members. Cases in which the previously generated feedback is not applicable 
are called “interventions”. We randomly sampled 3 out of the 82 exercises assigned in an undergraduate 
computer science course with 60 enrolled students. The exercises are labeled 1-3 in Table 1. Exercise 1 asked 
students to read in an integer n and sum up all integers from 1 to n. Exercise 2 involved reading from a text 
file and counting the occurrences of a given word. Exercise 3 was a simple OOP exercise where students had 
to create a “circle” class that has radius and area. Table 1 summarizes the results of the experiment. 


Table 1. Results of clustering experiment 


Exercise 

Num. of incorrect submissions 

Clusters by system 

Clusters by instructor 

Interventions needed 

1 

123 

28 

9 

3 

2 

180 

13 

9 

1 

3 

149 

14 

11 

0 


Despite the simple incorrectness-similarity metric used in this experiment. Table 1 shows that significant 
labor savings can be had. For example, the third exercise would have required 149 separate interactions 
between an instructor and a student, but the system reduced the number of required human-generated 
guidance passages to 14, resulting in a savings of over 90% with regard to instructor time. In addition, the 
system allowed all students who submitted incorrect solutions to be assisted in a timely manner, with most 
students receiving appropriate feedback instantaneously, all while requiring the instructor to do roughly 10% 
of the work that he/she would have had to do without the system. 


5. DISCUSSION AND CONCLUSIONS 

Our proposed e-learning system makes use of a case-based reasoning mechanism to improve the quality of 
feedback by looking at previously encountered incorrect submissions and the steps that were taken to resolve 
them. Once an instructor has determined the pedagogically appropriate feedback to remediate a particular 
misunderstanding, and that information is stored in the case base, students who share that misunderstanding 
will receive useful guidance immediately. In this way, our proposed system combines the rapidity of 
feedback delivery present in modern e-learning systems with the integration of human teaching expertise, 
which is used to design rich and detailed feedback. 

A metric that captures similarity with respect to incorrectness is an essential component of our CBR 
system. This metric allows the system to find previously submitted incorrect solutions that have similar 
flaws. Identifying a good similarity metric for this process is not easy. While one might imagine assessing 
similarity by performing some form of direct comparison of source code, we have found such approaches to 
contain hidden problems. First, surface features of source code, such as the names of identifiers, are typically 
irrelevant to the nature of the errors in a program. Minimally, some analysis of the logical structure of the 
source code samples must be done. There are existing tools for performing comparisons of the logical 
structure of two source code samples, but we have yet to find a way to leverage these tools for identifying 
common errors. For example, the popular MOSS system for detecting plagiarism in computer programming 
assignments does an excellent job of identifying pairs of programs that pass a threshold of similarity, 
suggesting a common origin for the two programs (Schleimer, S. et al, 2003). Upon careful examination of 
this system, we discovered that the kinds of comparisons needed to detect plagiarism are nothing like those 
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needed to detect common errors. Consider two programs that take very different approaches to solving a 
problem (e.g., an iterative approach versus a recursive approach), but make a common error in some small 
section of code, such as failing to initialize a critical variable. In this case, MOSS would rate the two 
programs as highly dissimilar, despite the fact that they share a common error. Conversely, consider two 
programs that are virtually identical, except for the fact that one makes a particular error in one line of code 
while the other does not. In this case, MOSS would flag the two programs as highly similar, despite the fact 
that they do not share an error. If the logical structure of source code is to be used as part of the similarity 
measure for the proposed CBR system, it will need to be carefully crafted to focus on similarities in terms of 
common errors. One possible approach would involve allowing human instructors to annotate source code 
submissions, identifying the sections that are most relevant for the detected error. 

In addition to exploring approaches to similarity focusing on such a static analysis of code, we intend to 
continue to investigate similarity metrics based on the common failure of programs on carefully crafted 
arrays of test cases. Possibilities include methods for automatically checking that individual test cases are 
discriminative of particular errors, comparing the output generated by test cases in more detail, and applying 
additional machine learning methods to learn to discriminate between error types based on a variety of both 
static and dynamic features. Since some errors may mask the presence of others, we are also designing 
methods for maintaining a hierarchy of potential errors, keeping the similarity metric from being deceived by 
such masking. 

Computers are ubiquitous in higher education institutions, and they are increasingly being used to 
improve the quality of the education being offered. E-learning systems have the ability to offer students 
immediate feedback, but it is often of the form of “Correct/Incorrect”. While such binary feedback is helpful, 
students may need additional guidance. We propose a case-based reasoning approach in order to try to 
improve the quality of feedback generated by automatic grading systems. We conducted a proof-of-concept 
experiment to determine the usefulness of CBR in a Computer Science e-learning system. Despite using a 
very simple incorrectness-similarity metric, our results are promising. Research is currently underway to 
improve the incorrectness similarity metric and to test the complete system in live classroom environments. 
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ABSTRACT 

Requiring students to complete their course assignments in partnership and in collaboration with students from other 
institutions is not commonplace teaching pedagogy. Even less so when they transcend disciplines and international 
borders. This Paper presents a brief account of an ongoing collaborative effort between Ryerson University, Coventry 
University and Loughborough University to inculcate cross border communication and teamwork skills to their Built 
Environment undergraduate students by way of having them work collaboratively on joint project assignments. It 
describes its scope and organisation and summarises some circumstantial and anecdotal observations of participating 
students’ inclination and disposition to working inter-institutionally. In an industry where cross disciplinary interactions 
and exchanges are the norm and where contracting parties to the project can frequently be across international divides, it 
is imperative that its professionals be trained in cross border teamwork skills. 


KEYWORDS 

Collaborative Learning, Built Environment, Construction Management. 


1. INTRODUCTION 

This student and faculty collaboration was prompted by two reports from The Royal Academy of 
Engineering, United Kingdom. The first is titled “Educating Engineers for the 21st Century” (RAE, June 
2007). The report took the stance that today’s business environment demands that engineers must have “the 
ability to work in globally dispersed teams across different time zones and cultures.” We surmise that the 
profession of engineers in this RAE report embraces all Built Environment (BE) professionals and in 
particular construction project managers and coordinators, whose background and training are commonly in 
related disciplines such as architecture and construction project management. 

Today’s Built Environment professionals and stakeholders need to be acutely aware of an increasingly 
globalised world where few economic borders remain. An economic “one-world” has bearings on 
procurement of goods and services in the BE industry as it can take away work that traditionally was viewed 
unmovable or “un-outsourceable” as well as creating new opportunities where there otherwise would be 
none. A good example is the growing use of offshore detailing firms for all manners of shop drawings and 
drafting services. These highly price -competitive detailing firms are largely located in Asia where a near 
half-day time difference oftentimes adds to their already competitive edge. 

If they seek to lead the industry tomorrow, today’s BE professionals must look beyond construction site 
boundaries. They must concede that they no longer have to sit in adjacent cubicles to get the work done, but 
more importantly they must learn not to. Advancements in information and communication technology over 
the last two decades have largely eliminated that need to. Tele-conferencing application tools such as Skype 
and GoToMeeting are nearly as physical as one can get to - minus the physical handshake. All manners of 
documents, drawings and renderings can be transmitted across the globe in mere seconds. Then there’s also 
the largest single form of communication in existence today - emails. Plus the modern telephone where a 
trans-Atlantic call can be made very inexpensively and oftentimes for next to nothing using one of the 
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bewildering numbers of Apps available. Use of some of these comes with concerns and trepidations but it is 
not the intent of this paper to discuss them. 

In its second report “Engineering Graduates for Industry” (RAE, February 2010), the academy apprised 
that academia must supply graduates with skills that ensure their “employability” meaning graduates with 
personal and interpersonal skills, communication skills, self-management skills, and skills in the application 
of information technology. Being fully equipped with mainstream technical knowledge and skills is no longer 
sufficient for today’s industry demand. None of these employability skills are more needed than in the BE 
industry. Few undergraduate curriculums in architecture and engineering schools have room for all of these 
skills. Most were learnt on the job, some painfully. 

What these all point to is a necessity for the BE professional to surpass an ability to engineer and to 
manage the complexities and uncertainties that are characteristically inherent in construction projects. To the 
best of our knowledge, there is currently a dearth of endeavour in BE professionals’ training programs where 
these two RAE’s advices are taken heed. Work in this regard started in 2011 with 37 Architectural Science 
students from Ryerson University in Canada and 32 Civil Engineering students from Coventry University, 
United Kingdom. Loughborough University, United Kingdom joined the collaboration in 2013, involving to 
date a total of 182 students from the three institutions. The intent of this paper is to share our experiences 
with educators our approach to address the two issues raised in the two RAE reports and outline what we 
circumstantially and anecdotally observed in the classrooms. It should assist other BE educators who may 
wish to implement similar collaborative projects in their own institutions. 


2. PROJECT SCOPE AND ORGANISATION 

This faculty and student collaboration stemmed from Coventry University winning a research grant from 
Hewlett Packard in 2010 to develop new and novel approaches to science, technology, engineering and 
mathematics (STEM) education. In its first year, 4th Year Architectural Science (Project Management 
Option) students from Ryerson University worked co-operatively and collaboratively with Coventry 
University’s 3rd Year Civil Engineering and Civil and Structural Engineering students on a year-long 
Design-and-Build project. Each project team consisted about equal numbers of students from each institution. 
The project brief was developed jointly by faculty instructors and took into considerations the institutions’ 
course requirements. At the end of the joint project assignment, each student was required to go online and 
assess the effort and contribution made by other team members using the Web-PA system. This confidential 
assessment returned a factor which was then used to adjust their earned final grades. 

In its second year, the project was changed to an A&A (Addition and Alteration) work to an existing 
academic building where unfettered access was available only during prescribed periods. It was felt that an 
A&A work as such would be more thought-provoking and challenging such as in designing, planning, 
scheduling and executing the work. Besides, it would require them to do some serious thinking around the 
context of the building project, including local building regulations and health and safety considerations for 
users of the building whilst it is being upgraded. The same project was used a second time in 2013 when 
Loughborough University brought to the collaboration its first group of 24 Final Year Construction 
Management students followed by a second group of 28 MSc Construction Management students in early 
2014. 

Project team composition thus closely fulfils the criterion for “globally dispersed teams across different 
time zones and cultures” except that the cultures of Canada and the United Kingdom are not generally 
recognised as significantly different. The teams, comprising architecture, civil engineering and construction 
management students are furthermore multi-disciplinary and to a good extent typifies project teams in the 
real world. Work was to be responsibly and equitably divided among the team members and was expected to 
be carried out in a coordinated industry manner. 

At the beginning of each academic semester, a faculty member from Coventry University would fly into 
Toronto to brief and take questions from Ryerson University students just so that students from both 
institutions received the same information and directions. This took place without fail since 2011 when the 
collaboration began, up till and including last year when Loughborough University joined. Further and 
additional briefings were conducted using Skype and latterly using GoToMeeting. 


390 



International Conference e-Learning 2014 


At team level, leadership was rotated every four to five weeks among the team members. The Team 
Leader is assisted by a Team Manager, whose main responsibility is to schedule and keep records of 
meetings, thus enabling the team leader to focus on coordinating team members’ progress and performance 
on their assigned sections of the work. At the succeeding leadership change, the manager assumes the role of 
team leader. At all times, we ensure that the team leader and the manager were not from the same institution. 

Each team identified itself as a company and prepared an initial proposal in response to a project brief 
developed jointly by faculty members. A well-thought proposal must include a design proposal to meet the 
client’s requirements, a set of specifications, a schedule and a preliminary cost estimate. Faculty members, in 
addition to advising and directing the teams, also doubled as clients. The assignment ends with a presentation 
to client and would typically include a reasonably detailed sets of architectural and structural drawings, a 
more definitive set of specifications, a revised cost estimate, a construction schedule and a method statement 
taking into consideration health and safety considerations, and for Academic Year 2012/13 a sustainability 
report on the proposal. 

Until 2013 when a grant for the project became available from the Higher Education Academy (HEA), 
United Kingdom, students communicate largely using Skype and emails. All manner of drawings and 
documents were deposited in Dropbox for further work. During the first year, the work was supported by a 
Hewlett Packard grant and students were encouraged to use HP Virtual Room communication software. The 
grant also made available use of HP laptops to Coventry University students. In 2013, GoToMeetings 
became the primary means of communication and work among students from the three institutions. This 
desktop sharing service allows the students to work together for their architectural and structural designs with 
3 dimensional Building Information Model (BIM) in real time with synchronous audio and video. The 
funding from HEA has permitted a higher level BIM collaboration to be implemented in this initiative. BIM 
has recently become the key requirement of building design and construction around the world. 


3. CIRCUMSTANTIAL AND ANECDOTAL OBSERVATIONS 

Working with geographically distant team members presented the students with some challenges and did 
require some changes and adjustments to the way they would normally do their work. The observations made 
here are largely based on casual conversations with the students and discussions among the students 
themselves over the years. At the time of writing this paper, work is ongoing to gain an understanding of 
some of the underlying reasons and motivations. We have, earlier, published those that we have conclusions 
(Soetanto et al, 2012; Soetanto et al, 2014; Soetanto et al, 2014). This paper does not intend to discuss them. 

3.1 Time Difference 

Time wise, the United Kingdom is ahead of Canada by 5 hours which effectively means that when the 
Canadian students start their class hours (invariably after lunch time), their United Kingdom team members 
would likely have been tired out from a full day at school. A 5-hour time difference is significant in that it 
effectively leaves only a common 3 work hours from a typical 8-hour work day. Canada moves to Daylight 
Saving Time typically 2 weeks earlier than the UK and a resulting time difference of 4 hours sometimes 
caused confusions with scheduled meeting times. But for the most part, the students managed well such as 
agreeing on days and times for meetings and for working together. Weekends seemed to be a popular choice 
of days when most have no classes to attend. 

3.2 Multidisciplinary Aspect 

In contrast, it was the multi-disciplinary aspect of the collaboration that seemed to be more difficult for the 
students to deal with. Whilst we observed numerous constructive discussions among the architecture and 
civil engineering students as to what works and what doesn’t, we also observed that at times they find it hard 
to come to terms that each discipline has particular strengths (and weaknesses). For example, the architecture 
students have on a number of times lamented on their counterparts’ lack of design aesthetics and drafting 
skills. Another issue which sometimes cropped up relates to common knowledge that all the students would 
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be expected to know by then but have yet to be taught or in the process of being taught - such as use of 
drafting and scheduling software. 

3.3 Team Members Preference 

There is little doubt that if they had a choice most of the students would prefer to team up with friends and 
with those they knew in the class. In this collaboration, students first formed “sub-teams” of their choosing 
from the class - typically of two to three students. Faculty members then randomly ascribed one sub-team 
from each institution to form a project team of seven to eight members. Thus, they do get to work with a few 
team members they know but with more they do not - a situation most Built Environment professionals will 
find themselves in the real world. It is not clear to us why this would be so and we conjecture that familiarity, 
perceived reliability or otherwise, competence or simply just plain chemistry may play a part. When asked 
late into the project, most valued their experiences of working in an international multi-disciplinary team. 

3.4 Team Interactions 

On the whole when the students learned that they would be required to work with students from other 
institutions, about half of them displayed some unease - such as what their team members from the other 
institutions might be like in terms of technical competence, diligence, work ethics and responsibility. These 
trepidations either diminished or get exacerbated into the project. Over the three years, about one in five or 
six groups did not get passed their disagreements and differences - consistently about work below 
expectations or not on time, ineptness on the part of some individuals, unreliability, not attending meetings 
and, in a few instances, non-participation or apathy. But for the majority, team performance and cooperation 
were as best as be expected and the quantity and quality of work produced were very good. 

Referring to the other groups of Coventry students who were not participating in this international 
collaboration, one testimony suggested that working with distant partners online was in fact sometimes better 
than working offline. Some students have also mentioned that they work harder as they are representing their 
university and don’t want to be seen in a ‘bad light’ compared to other institutions. With other supporting 
evidence, it could be concluded that successful collaboration is very much dependent on the professional 
work ethics and trust. The impact of mediating technology is not critical if the team has strong work ethic and 
trust (Soetanto et al. 2014). 

3.5 Us and Them 

Whenever work was not progressing as planned, the “us and them” differentiation or discrimination would 
come into play with little hesitation that the culpability rests squarely on “them”. Notwithstanding being 
reminded time and again that there is no “us” and “them” in a team, this mindset persisted. Over the years, 
we have not had one single complaint where a student placed the fault or shortcoming on the team member 
from his or her own class. This distinction between us and them embraced issues such as design and drafting 
skills, ability to write and structure a report, or simply “we are doing more work than them and it’s not fair”. 
What was also noticeable over the years was that students tend to rate their contributions higher than those of 
their virtual team members in terms of quality and quantity. 

3.6 Recognition and Fair Play 

Notwithstanding the “us” versus “them” mindset that permeated, students were also quick to give recognition 
and acknowledgement to their virtual team members and this was commonly reflected in the end of project 
Web-PA peer evaluation that was also used to adjust their earned grades. Over the years, we have also heard 
frequent complimentary and admirable comments from students about some of their virtual team members. 
Likewise, they are also quick to castigate those whom they felt had not lived up to their expectations. They 
would also not hesitate to use the confidential peer evaluation process to rate them negatively. Deemed “free 
loaders” appeared to be the worst categorisation. 
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4. ONGOING WORK 

Work is continuing with the collection and analysis of student experiences and the issues encountered - both 
students interaction-related and technology-related. The findings will be used to develop a guidance of 
effective practices for international student collaboration in a real-time online platform. Experiences, cases 
and lessons learnt will feature in the guidance. 

An online BIM-Hub is currently under development to assess approaches, practices and technologies to 
support this international student collaboration, concurrent with its use as a platform for student virtual 
collaboration. When fully completed, it will encourage a community of learning among HE academics and 
their students through open discussion forums in relation to issues such as experiences and practices. The 
BIM-Hub will also provide limited access to interested external parties, support a range of social media 
popular among students and will be supported beyond the life of this project. 


5. CONCLUSIONS 

Increasing international collaboration in the Built Environment industry obliges its professionals to have “the 
ability to work in globally dispersed teams across different time zones and cultures”. This skill set embraces 
skills to interact and to work with counterparts from distant lands employing appropriate communication 
technologies, which no doubt enhance their employability in the industry. 

In addition to their (design and technical) program requirements, this collaborative effort among our three 
HEIs provided our students an opportunity to learn and practice team work skills in an online environment 
across geographical and time zones divide. The multi-disciplinary team environment challenges them to a 
range of issues that are not uncommon in the real world. And they also get to acquaint themselves with the 
industry practices of another country. We are of the view that as BE educators we have an obligation to 
inculcate these skill sets in tomorrow’s Built Environment professionals. 
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ABSTRACT 

This short paper describes an ongoing NSF-funded project on enhancing science and engineering education using the 
latest technology. More specifically, the project aims at developing an interactive learning system with Microsoft 
Kinect™ and UnitySD game engine. This system promotes active, rather than passive, learning by employing embodied 
cognition with interactive visual/spatial information, in which human movements could impact a lasting effect on both 
the short-term episodic and long-term memories of students. Two groups of Biomedical Engineering students at The 
University of Akron participated in a study of the four educational modules (Cell, DNA, Immune System, and Gene 
Therapy) and were tested after the conclusion of each educational session. The preliminary results show a trend toward 
better performance with the system compared to traditional instruction in second-year college students. However, more 
studies with a larger group and with a younger population, such as K-12, need to be evaluated since these students have a 
less developed visuo-spatial sketchpad relative to the sophomores in college. 
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1. INTRODUCTION 

When teaching educational contents that are inherently graphical in nature, research has shown that learning 
is more efficient when stimulating the visual component of the brain (the visuo-spatial sketchpad) as 
compared to the language -based approaches [1]. In the working memory model proposed by Baddeley and 
Hitch [2], the central executive controls the flow of information from and to the phonological loop (verbal 
buffer), the visuo-spatial sketchpad (visual buffer), and the episodic buffer (where context is added to short- 
term-memory codes for storage in long-term memory). We know that both verbal and visual codes are stored 
in short-term memory because we can use these two types of codes to interfere with working memory 
performance [2]. Because of the “video game effect,” there is accumulating evidence that individuals can use 
visual codes to help supplement verbal-code information in short-term or working memory. 

Furthermore, there is evidence that humans use “embodied cognition” to improve performance [3-5], 

That is, we encode the state of the body and use it as a “somatic marker” (or contextual cue) when storing 
working-memory information in long-term memory [3-6]. Using these assumptions, it is predicted that 
visual codes and motoric feedback from movements should improve the educational outcomes in learning 
science and engineering information. 
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Figure 1. Components of Baddeley and Hitch’s updated concept of working memory. 

The goal of this project is to refocus science and engineering education by introducing a new technology 
that is integrated with interactive, educational content that engages, inspires, and stimulates the visual center 
of the brain. To accomplish these goals, a new curricula has be designed and implemented that engages 
students and interactively introduce visual information using the Microsoft Kinect™ device [7], This device 
will be coupled to a computer system and will allow students, as an example, to actively explore a cell’s 
anatomy and its functions using an interactive spatial interface instead of simply showing pictures with 
verbal descriptions. This approach also promotes active, rather than passive, learning by employing 
embodied cognition, where human movements could impact a lasting effect on both the short-term episodic 
and long-term memories of students [3-6], In our approach, both the visual/spatial information and the 
motoric interactions can be stored with a memory trace of the learning event (e.g., structures and functions of 
an eukaryotic cell and its organelles). 

We hypothesize that introducing graphical information that is designed specifically to simulate the visuo- 
spatical sketchpad with gestures will positively impact the students’ learning and have a lasting effect on 
cognition and episodic memory [8-9]. Since our educational content is also designed similar to a game 
interface, we will also enhance the emotional states of the students which is associated with potentially better 
episodic memory performance for remembering events [9] as demonstrated by applications of Damasio’s 
somatic marker hypothesis [3-5], Thus, encoding the visual information (in visuo-spatial sketchpad), along 
with integrated gestures, should result in more accurate retrieval of the contextual information than will lead 
to more efficient learning. 


2. SYSTEM DESIGN 

Human gesture recognition is an important part of advancement in human -computer interaction [10]. Recent 
development in natural user interface (NUI) is exemplified by gesture-based interactions. Natural user 
interfaces are user-centric and allow people to interact with the computer in ways that are easier to learn. By 
eliminating the mediator of a keyboard or mouse between the user and the computer, gesture -based 
interactions enable the user to directly interact with the visual content on the computer. Furthermore, if the 
users are represented as an avatar, they will gain the sense of embodiment and be immersed into the 
programmed contents. 

Kinect™ device is the latest natural user interface that provides real-time gesture recognition [11]. In 
addition to a RGB color camera, it has a depth camera, which consists of an infrared projector and an infrared 
sensor [12-14]. Microsoft® has developed an algorithm that utilizes these cameras to recognize human poses 
[15], Furthermore, the depth information enables the system to capture gesture motions in three dimensions; 
therefore, Kinect™ recognizes left-right, up-down, and front-back motions [15]. 

In this project, we use Kinect™ to capture the motion of students and track their gestures to allow them 
visually interact with the education contents. We developed a set of educational contents for Biomedical 
Engineering and created virtual 3D objects for four learning modules: Cell, DNA, Immune System and Gene 
Therapy. Those virtual objects are then imported into a game engine (Unity3D [16]) for animation and 
display. User movements captured by Kinect™ are sent to the game engine through the application- 
programming interface (API) provided by (OpenNI [17] and Zigfu [18]). The students will be able to rotate 
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the virtual objects, for example, the entire cell or its organelles. The students will also be able to zoom onto 
nano-sized molecules such as DNA, ribosomes, and cellular receptors. They will be able to study not only 
their structures but also their functions through interactive animation. They will have the control to 
translocate from outside of the cell into various organelles within the cells. This “first person” perspective 
will use embodied cognition and should import the relevant contextual information more efficiently into the 
student’s long-term memory. These and other features will be linked to the student’s gestures that are 
captured by the Kinect™ system. Since these gestures are intuitive, the leaning curve to use the proposed 
educational modules and the hardware will be minor. 

Figure 2 depicts the architecture of our Kinect™ based system and figure 3 demonstrates the system in 
action. 



Figure 2. The architecture of the Kinect™ based learning system. 


Figure 3. The system in action. 


3. EXPERIMENT DESIGN 

An experimental goal of the project is to compare the learning outcomes for seniors and sophomores 
undergraduate students using traditional verses technology-enhanced (Kinect™) teaching methods in 
biomedical engineering courses. The rationale for this preliminary experimental design is to compare 
populations of students with fully developed visuo-spatial sketchpad (seniors) to a group in which this 
component of their brains is still under development (sophomores). Thus, seniors are more efficient in 
learning by visual means. The learning and understanding of educational materials related to each module 
will be tested. The learning and understanding of educational materials related to each module will be tested. 
The experimental design is given in Table 1. Students will be exposed to both the traditional techniques 
(through modules 2 and 3) and enhanced teaching using the Kinect™ system (through modules 1 and 4). The 
contents for Modules 1-3 are focused upon biological science. In contrast, module 4 is focused upon 
biomedical engineering. This experimental design will allow for intragroup and intergroup comparisons. In 
this way, all students will be exposed to our Kinect™ system. After each session, students were tested. The 
results for the traditional and technology -enhanced teaching methods were averaged and compared using 
paired t-test. 

Table 1 . Experiment groups tested using traditional verses enhance teaching methods. 



Seniors 

Sophomores 

Module 1 - Cell 

Kinect™ 

Kinect™ 

Module 2 - DNA 

Traditional 

Traditional 

Module 3 - Immune System 

Traditional 

Traditional 

Module 4 - Gene Therapy 

Kinect™ 

Kinect™ 
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4. PRELIMINARY RESULTS AND FUTURE WORK 

Two groups of Biomedical Engineering students at the University of Akron participated in study of the four 
educational modules (Cell, DNA, Immune System, and Gene Therapy) and tested after the conclusion of 
each educational session. We expect that seniors using the Kinect™ system will not perform better with 
technology-enhanced teaching method, and the preliminary results show that no differences for seniors using 
both the Kinect™ and traditional approaches (Figure 3, p = 0.92). For sophomores using the Kinect™ 
system, their performance in learning is better with the technology-enhanced method, but the results were not 
statically different (p = 0.49) from the traditional method. However, a correlation exists between the 
Kinect™ and the traditional method for this group (p = 0.034). With n = 6 and 5, respectively, for each 
group, we demonstrated the potential utility of the Kinect™ system’s ability to enhance teaching graphical 
information. However, more studies with a larger group and with a younger population, such as K-12, need 
to be evaluated since these students have a less developed visuo-spatial sketchpad relative to the sophomores 
in college. 



Figure 4. Results of the Kinect™ and the traditional teaching methods for seniors and sophomores. 
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ABSTRACT 

Digital technologies are revolutionizing all parts of society, including higher education. Universities are rapidly adapting 
to the prevalence of staff and student mobile devices, digital tools and services on campus, and are developing strategies 
to harness these technologies to enhance student learning. In this paper, I explore the use of digital technologies to 
support blended learning in universities, and discuss how massive open online courses (MOOCs) can be improved 
through better understanding of successful use of technology, communication and collaboration in such scenarios. 


KEYWORDS 

Blended learning, higher education, learning, digital technology, online courses, students. 


1. INTRODUCTION 

The academic community has largely accepted PowerPoint and Virtual Learning Environments essential 
tools of the trade, but we face new opportunities and challenges. For example, our students are bringing 
mobile devices into the classroom and using them to access information and communicate, and we have 
lecture capture systems recording our classes. Massive Open Online Courses (MOOCs) are available to our 
students to supplement their learning and personalised learning environments and use of learning analytics 
are set to transform education. There are expectations from our students of a blended education and 
expectations from governments and policy makers of increased integration of technology in our approach to 
education. All of these issues provoke vociferous discussion in academic settings, as we all grapple with the 
rapidly changing landscape of learning in a digital age. 

In this paper, I will explore the nature of these changes, the implications for teachers and students and 
the challenges and benefits of a holistic approach to digital learning for a modern university. I will also look 
forward to the future of education in our connected society. 


2. THE ERA OF THE DIGITAL STRATEGY 

The nature of student education is changing. Information is easily accessible via the Internet on PCs, laptops 
and mobile devices, and the challenge for teachers has shifted from one of information transfer to a role 
supporting students to curate, filter and critique information, and use it to solve real-life problems. This shift 
has been rapid and unprecedented. The notion of the lecturer as the custodian of knowledge is hard to 
overturn, particularly when the alternative means re-thinking curriculum design, teaching delivery, student 
activities, assessment and feedback. But we must adapt the ways that students are taught and learn in Higher 
Education as this will impact on their skills, competencies and ability to contribute effectively in a digital 
workplace and society. The world is now digital and higher education must reflect this, and indeed should be 
at the vanguard of the revolution leading the way for other sectors to follow. 

Many universities are recognizing (at a senior level) the need to revolutionize higher education in this 
changing landscape, with many introducing ‘e-learning’, ‘blended learning’ and more recently ‘digital’ 
strategies to support the changes in curriculum design, academic practice, infrastructure and training and 
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support necessary to deliver higher education in a digital age (Garrison & Kanuka 2004). Take for example 
the author’s institution, which has a ‘Digital strategy for student education’ and a ‘Blended learning strategy’; 
the former sets the overall agenda for the digital ecosystem at the University, prioritizing investment in 
seamless, user-focused content, tools and services to support education and student services. The blended 
learning strategy sets the pedagogical framework for appropriate use of online resources, technologies and 
tools to support face-to-face interaction between students and teachers. In combination, these strategies 
position the University as an open, content-rich, digitally connected environment where face-to-face 
interaction is supported by online multimedia content, digital tools and services. In this environment, 
teaching spaces are digitally equipped, and the VLE is just one of a range of online tools for providing 
students with learning resources and opportunities for interaction and collaboration. External digital learning 
channels are well populated with openly available content, and the University is offering the full range of 
face-to-face, blended and fully online courses. 


3. BLENDED LEARNING APPROACHES 

Staff in universities are using an increasing number of digital approaches to support student learning, drawing 
on a growing pedagogical literature evidencing the effectiveness of technology to enhance learning 
outcomes, student engagement and enjoyment (Sharpe et al. 2006). The following approaches are growing in 
use within universities by both staff and students: event capture, mobile learning, mobile voting, eBooks, 
social media, e-assessment and use of open educational resources. Each has its own literature base 
expounding the benefits and challenges of adoption for teachers and students, and all are supported by a 
plethora of technical solutions. As we continue to understand the potential of the connected web to support 
and enhance learning, students and teachers will expand their use of participatory, collaborative and 
networked learning approaches. 

The availability of these blended learning approaches has reignited the debate about the most appropriate 
sequence of events when supporting students to acquire new knowledge, skills or competencies. Advances in 
technology and online repositories have made the concept of the ‘flipped classroom’ a realistic and viable 
option for the majority of educators (Sams & Bergmann 2013). In essence, the flipped classroom overturns 
the traditional sequence of events and requires students to engage in conceptual understanding of key facts 
and information using online resources and tools prior to interacting with educators and learning mentors. 
Imagine a course where the traditional ‘lectures’ have been digitized using an event capture system, edited 
into chunks or bite sized learning units and provided to students online with some introductory materials, 
instructions and opportunities for online interaction and discussion between learners. The original lecture 
time is replaced with tutorial or seminar style sessions where students undertake problem solving, discussion, 
Q&A or other collaborative activities using mobile devices, to reinforce understanding, challenge 
misconceptions and deepen knowledge; a very active learning approach. 

A holistic, seamless and well-integrated blended learning approach using pedagogically appropriate tools 
can enhance student learning and the quality of the student experience. Combining the flipped classroom 
approach with integrated use of mobile devices containing multimedia interactive eBooks and apps to support 
learning, along with access to online learning resources and tools for student collaboration and interactivity 
would satisfy many educators and students. However, this solution brings with it a number of challenges, 
including digital fluency, technical skills and factors affecting the adoption of technology, including 
perceived ease of use and usefulness (Venkatesh et al. 2003). 


4. HOW MOOCS CAN LEARN FROM BLENDED LEARNING 

Since 2008, Massive Open Online Courses (MOOCs) have dominated the higher education headlines, with 
promises of educational revolution and global learning, and worries about the end of physical universities, an 
abundance of cheating and plagiarism, and superficial education. None of these prophecies have yet 
materialized, but MOOCs offered on the major xMOOC platforms (e.g. Coursera, Edx, Futurelearn) have 
gained a seat at the table of higher education, as one of a number of routes available to learners, alongside 
blended learning, face-to-face and traditional distance courses. However, there is room for considerable 
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improvement in the pedagogy of massive open online courses as they evolve and morph (Siemens 2013), 
drawing on the experience gained in distance education and blended learning. 

MOOCs that engage, stimulate and provoke learners to meet their learning goals require consideration 
and understanding of the following elements: (1) course design, navigation and accessibility (learning 
design); (2) participants’ learning goals, pre-requisite knowledge and digital literacy skills (learner skills) (3) 
opportunities for interactivity, communication and collaboration between learners and with subject experts 
within the course (social learning); and (4) the methods of assessment and feedback (learner outcomes). 
These are critical elements of well -designed traditional face-to-face, blended and online courses, but there is 
one key difference: MOOCs attract an audience which is often not predefined, from 16 year old school 
students, current undergraduate and postgraduate students, through to professionals and leisure learners. 
MOOC participants are all at different levels trying to reach a clear learning goal from the same materials 
within a defined learner journey. 

It has become clear that the majority of MOOCs attract well-qualified, professionals seeking to learn 
something new. However, most courses have very low retention figures with high drop-out rates early in the 
course (Koller et al. 2013), which probably results from a combination of the following factors: (i) curious 
learners with no intention to complete, (ii) learners who only needed or wanted one aspect of the material to 
meet their learning goal, (iii) learners with good intentions but lack of time or commitment to study online, 
(iv) disengaging learners who were taught in a face-to-face environment who struggle to transition to an 
online environment and (v) learners who do not respond well to the teaching style (be that didactic or 
participatory). Clearly, future online courses will need to address these issues to encourage a greater 
proportion of participants to stick with the course through to the end, in order to achieve the course’s learning 
outcomes. It is worth noting that most face-to-face, blended and wholly online courses normally offer 
learners support to make the most effective use of the resources e.g. course map, learning journey etc. and 
provide study skills advice about how to study effectively online e.g. how to communicate, how to learn from 
multimedia resources, how to provide peer feedback etc. (Morris et al. 2014). The majority of MOOCs do 
not offer this support to learners, and it may be a contributing factor in the large dropout rates observed. 

The constructivist or ‘c’ MOOC encourages learner centered, non-linear learning experiences where 
problems, ideas and solutions are developed through sharing the best of the web, creating materials and 
communicating through a variety of channels, including social media networks (Brennan 2013). However, 
the cMOOC is seen as inaccessible or daunting to some learners, particularly those lacking expertise in the 
subject or lacking confidence and skills in a digital environment. The answer, the xMOOC, where the focus 
is on tutor-generated content, a linear learning experience and more structured opportunities for 
communication, appeals more to the masses, but may not offer the best deep learning experience, due to lack 
of active and participatory learning. 

It is rapidly becoming apparent that the distinctions of c- and xMOOC are unhelpful, and the challenge to 
MOOC developers is to offer the best of both scenarios, so that the subject expert and the learners contribute 
problems, ideas and solutions equally, using the best of the web and through creation of digital materials, and 
offering opportunities for rich conversation in structured, accessible and appropriate ways for learners with 
differing needs. Conversation drives learning, through clarifying understanding, problem-solving, building 
mental representations and fostering deep learning (Laurillard 2002), and this must be better represented in 
online courses. The challenge for MOOCs, and indeed any online learning opportunity, is to provide learners 
with regular, authentic, accessible, and structured opportunities to engage in conversation with other learners 
about the course materials supported by subject experts. 


5. FUTURE OF ONLINE LEARNING 

Laurillard has recently argued that learning technologies are “hopelessly underexplored” (Laurillard 2014), 
due to the fact that most online learning experiences don’t take account of the theories that underpin effective 
education. Therefore, we should now begin to explore the potential of learning technologies to offer a 
participatory, active, networked and personalized online learning experience that delivers real improvements 
in learning for learners at all levels, and with differing learning needs. 

MOOCs could offer high quality learning experiences for all types of learners, but at present they are too 
linear and one dimensional to fit the needs of all participants. So, let’s consider a MOOC that adapts to the 
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individual. In order for this to happen, the platform will need to understand the learner in advance of the 
course. This information could come from a pre-course survey, in which the participant reports on the 
experiences, prior modes of learning, social learning preferences etc., or the system could interpret their 
behaviors from analysis of their social networks, if the participant provides account details for Facebook, 
Twitter, Google etc. upon registration. Algorithmic and social network analysis could be used to predict if 
the participant is likely to prefer a teacher-focused or learner-focused experience, adjusting their participatory 
exercises accordingly. Once the course has begun, learner analytics should dynamically adjust the learning 
experience, based on the participants’ behavior (e.g. choice of media format, page navigation, dwell times, 
social activity), constantly checking with the participant that the changes are suitable. In this way, 
participants will receive a personalized and dynamic online learning experience suited to their preferences, 
which would be likely to enhance their engagement, enjoyment and ultimately the quality of their learning. 

Of course, this model doesn’t address the challenge of effective participatory and collaborative learning at 
scale, with thousands of learners and only a few tutors. Many have argued that this is a fundamental flaw of 
MOOCs, but given that many courses have well qualified professionals enrolled and participating, it may be 
possible to use their collective knowledge to mentor and support other learners, in a similar way that 
discussion forums use validated expert contributors to answer questions. 


6. CONCLUSION 

In this paper, I have explored the current use of digital technologies in blended learning approaches, and 
outlined their benefits and challenges. I have also attempted to suggest how massive open online courses 
could be improved, by drawing out proven benefits of blended learning which could be applied to such 
courses, and I have described how online courses might evolve in the future to capitalize on learner analytics 
and personalization. Of course, massive open online courses are only one route for learners to enrich their 
learning online, and they must evolve to cater for individuals with differing goals, background and skills if 
they are to have a long-term future for the masses. 
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ABSTRACT 

The growing influence of technologies on all aspects of life, including the education sector, requires developing countries 
to follow the example of the developed countries and adopt technology in their education systems. 

Libya has been able to boost its economic and educational position over the years, and this brings it to the concern of 
applying modern methods of learning into its Higher Education (HE) system. E-Learning has been advocated by several 
university professors and researchers as one of pertinent method in the education system in the current society context of 
busy schedules and dual responsibilities of adult learners. However, due to many cultural, governmental and 
technological reasons, the state of e-Learning in Libya’s HE has not been to an adequate level. 

This paper aims to shows the outcomes of several studies, which discover the effect of national and international ethos and 
methods to education on the usefulness of manipulating, applying and utilizing e-Leaming schemes and machinery in 
Libya’s Higher Education institutions. The writers have advanced an inquiry form, which was finished by ripe Libyan 
students registered for PhD educations in the United Kingdom, who are also permanent professors at the Universities of 
Tripoli, Garyounis, El-Zawia and Aljabal-Algharbi. The measurable and qualitative inquiry of the replies show numerous 
complications connected to the utilization of e-Leaming and, Information and Communication Technology (ICT) in Libyan 
universities. 


KEYWORDS 

E-Learning, Information Communication Technology (ICT) Infrastructure and Higher Education (HE). 


1. INTRODUCTION 

E-Leaming denotes to the utilization of electric media and information and communication technologies 
(ICT) in learning. E-Learning is widely comprehensive of all forms of instructive skill in learning and 
coaching. E-Learning is comprehensive of, and is widely identical with montage learning, technology 
enhanced learning (TEL), computer based instruction (CBI), computer based training (CBT), computer aided 
instruction or computer assisted instruction (CAI), internet based training (IBT), web based training (WBT), 
online learning, computer-generated education, virtual learning environments (VLE) (which are also named 
learning stages), m-learning, and digital educational association. These substitute names lay emphasis on a 
specific feature, constituent or distribution technique. 

E-Learning comprises of several kinds of mass media that convey text, aural, metaphors, animatronics, 
and flowing video, and comprises of technology uses and procedures such as audio or video tape, satellite 
television, CD-ROM, and computer grounded learning, as well as native intranet or extranet and web based 
education. Information and communication schemes, whether free vertical or grounded on either native nets 
or the Internet in schmoozed learning, underlie many e-Learning procedures. 

E-Leaming can befall in or out of classroom. It can be self-paced, asynchronous education or may be 
teacher commanded, synchronous education. E-Learning is suitable to remoteness education and plastic 
learning, but it can also be utilized in combination with face-to-face education, in which situation the term 
merged education is commonly utilized. 

Libya has a decent evaluation for learning in the setting of the Arab world, bestowing to the United 
Nation’s Human Progress Index. Libya remains at the topmost of the list of all African countries in provisions 
of education, not only geologically, but also tactically. Libya has enduringly been keen to safeguard entrance 
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to suitable education for all associates of its society, men and women. The administration system devices plans 
to develop and progress the ICT substructure of Libya, and it pursues to progress and refurbish the complete 
educational procedure, together with the enlargement of prospectuses and informing its logical content; the 
implementation of ICT within teaching, together with higher education, is an vital factor in its general 
development tactics. Al-badree (2006) converses the educational, technical, and attitudinal contests connected 
to this. The outline of e-Learning packages in the edifying system of a particular country must take into 
reflection the communal and cultural features of that humanization. The communal and cultural upbringing of 
the educationalists and beginners plays an important role in deciding the achievement of e-Learning teaching. 
This importance varies from one culture to another, according to the ethics of the society, and its civilizations 
and customs. 

E-Learning needs more than mere technology to be fruitful. There is the necessity for educational 
specialists who are well skilled in ICT, proficient of utilizing e-Learning systems and emerging learning 
constituents that discourse the needs of beginners. Nearby based experts are also mandatory to sustain 
apparatus as well as e-Learning schemes and tackles. 


2. E-LEARNING IN LIBYA 

Libya plays a chief role within the African landmass, on both an occupational and an educational level, by 
stimulating and supporting main creativities and plans, comprising of those in the adjacent states of Sudan, 
Chad, Niger and Mali. Nevertheless, the contests of a pitiable and unripe substructure, joined with a deficiency 
of experienced, skilled teachers, ICT endowment and ICT for teachers, shows a great trial to the present 
transformation process. The 1 st methodical study of the application of e-Learning schemes in Libya was 
directed by Al-badree in 2006 and this study signposted that the application procedure is still in its 
determinative years. The effort to inspect e-Learning is yet at a situation study phase, because the disposition 
of info and infrastructures skills is not extensive. Nevertheless, teachers have been appearing in drill courses 
on e-Learning operation since 2002, and e-Learning was successively combined into the HE exam process in 
2005. The Libyan Section of Education has lay emphasis on that ICT generates new methods of education & 
drill and has the ability to augment the organization of, and progress the level of, teaching in Libya. The 
worldwide spread of ICT has permitted individuals to utilize skill in all scopes of life, be it at work, at home, 
in institutes or in the field of amusement. This has controlled to an augmented number of beginners and 
coaches in Libyan universities, institutions and colleges bringing distance learning sequences. 

This paper will show the features that pupils reflect to be blockades to initial, on-going, and concluding 
online educational sequences, utilizing a quantifiable investigation of the replies to an investigation survey 
intended and applied by Kenan (2009). The analysis offered the views of sixty three accused, which consist of 
12 (19.05%) females and 51 (80.95%) males. The survey recognized the contests skilled by teachers, pupils 
and practical staff in HE organizations in Libya. The queries were articulated after an extensive initial study 
had been done, linked to barriers to the application and utilization of e-Learning and ICT in learning. The goal 
of the survey was to check if the accused views definite previous insights concerning the blockades to utilize 
e-Learning and ICT in learning. Also the accused were required to direct their individual opinions about any 
other contests they confronted when dealing with e-Learning and ICT. 

The education freight in Libyan universities is characteristically large; for instance, the normal number of 
coaching hours for educational staff is 24 hours per week, and Libyan universities have not up till now 
recognized a scientific investigation tradition. Consequently, even professors find it difficult to find the time 
for active research and educational development. The after graduate packages started in 1973 in several staffs 
of Libyan universities comprised of education (Tripoli University) and prose. In 2004, the entire number of 
after graduate pupils who had required a Master’s degree from Libyan institutes in diverse disciplines was 
about 3150, and only 40 students had acquired a PhD certificate from the 3 main institutes. Although 7 or 8 
institutes now have the incomes and educational position to award PhDs, many pupils, specifically in 
engineering, science, management and finance, found it essential to voyage abroad to carry out after graduate 
learning, and there were some 3500 who did this in the academic year 2004/2005 (Said, 2005). 
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2.1 The Challenges to E-Learning in Libya 

Artemi (2009) categorized the contests connected to the application and utilization of e-Learning and ICT in 
Libyan organizations into 3 classes: deficiency of ICT substructure; deficiency of skilled workforces, and 
confrontation to alteration. Kenan (2009) performed further studies regarding these contests. She clustered the 
barricades into 3 categories grounded on the deductions from her study, and on her private skill as an 
educationist: Organization barriers; Technical barriers and Social barriers. 

2.1.1 Technological 

The maximum proportion of professed blockades is knowledge, at 34%; this blockade has an effect on 
instructive procedures and is connected to: IT recital abilities; design abilities; boundaries connected to 
bandwidth, and the safety supplies of IT systems. The mainstream of Libyan beginners does not have access to 
a personal computer or the Internet. Libya is still way behind other countries in terms of access to private 
computers. In 2004 it was assessed that in Libya, personal computer density was low, at 3.4 per 100 people; 
nation-wide 17% of Libya schools had a processor, but only 12% had one for coaching and education 
(MBNQA 2004). While in other states, such as United States and United Kingdom, the fraction of computers 
accessible in subordinate institutes was 73% and 78% correspondingly (Consultation Unit 2007). 
Nevertheless, there are also many less advanced states where computer learning is very great. In accumulation, 
the capability to contact the Internet and the amount of Internet users varies extensively from country to 
country. 

2.1.2 Mismanagement 

The proportion given for misconduct as a blockade is 29%; there are numerous issues which oppose the 
operation and utilization of e-Learning and ICT in Libya, such as augmented amount of work for educational 
staff; advance time; conveyance time; lack of strategic arrangement and visualization; deficiency of drill in 
technical progresses, and lack of provision for educational features of the progresses. 

2.1.3 Cultural 

The proportion of accused who professed culture as an obstacle was 21%; cultural obstacles occur where a 
definite ethos or collection is incapable to admit or accept a new procedure in a significant area of their lives, 
due to features such as spiritual principles, social taxes or habits. This boldness has been strengthened by 
actions because, with the arrival of new technologies, jobs that could previously be done with a minimum of 
education have quickly disappeared. A main notion of e-Learning is the litheness of timing for pupils, but 
some religions enforce a stern daily schedule, and it is also extensively known that many institutions of higher 
education have timetables, which are stationary and not at all lithe. In order to reflect social issues or social 
encounters that could act as blockades to e-Learning, one has to find the causes why people or persons may 
favour not to study in an electrical atmosphere. Some of the causes are matters such as: the terror of signifying 
a deficiency of ability or capability; terror of skill; terror of separation from other pupils; deficiency of 
responsiveness of the necessity to progress or the chances obtainable; accusing others for insufficient 
presentation somewhat than captivating accountability for one’s own activities; deficiency of private self- 
assurance, and a overall belief that persons cannot alteration. Consequently, fear stances a solemn blockade to 
e-Learning, because it is merely through coverage and skill that one can expertise or be relaxed with e- 
Le anting. 

The ethos which still rules most investigation presented by Libyan institution of higher education is usually 
the notion of a ‘research drill’, such as expositions presented by scholars to acquire documentations, or by 
educational staff to whole the educational necessities for job advancement, so the aims of such investigation 
have not arisen from the actual requirements of civilization. The Libyan Business Administrative Survey or 
Global Keenness Report (LBES/GCR) positions Libya 97 th out of 1 1 1 states in institution of higher education 
or business investigation cooperation. Nonetheless, some educational staff conducts additional happenings, 
such as inscription and publication of text books for instance, to upsurge their salary. 
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2.2 The Benefits of Using E-Learning and ICT in Libyan Institutions 

Libyan institution of higher education could profit from the notion of lively education, and progress it in the 
shape of e-Learning, where scholars are not only hearers in the period, but also intermingle with the 
instructor and converse collected the information presented by the matter. Both lively education and 
e-Learning inspire students to use numerous bases of information, and encourage them to incorporate and 
occupation info proficiently, so that scholars are permitted to generate queries and converse new notions 
inside functioning teams, where info is joint in order to attaining a mutual aim. 


3. CONCLUSION 

E-Learning can prove creative in meeting the challenges of higher education. In Libya, e-Learning has 
enormous prospects. The vast and constant improvement of the information and telecommunication 
technologies in Libya indicates that society is ready to accept and embrace e-Learning fully. However, 
special care should be taken to analyse the opportunities and factors that can influence e-Learning adoption 
and implementation. 

This paper reflected on the higher education context in Libya and the applications of ICT and e-Learning 
in Libyan higher education to date; it also discussed the challenges for and prospects of integrating ICT in 
higher learning institutions in Libya. It was found that the challenges or the barriers to e-Learning in Libya 
could be classified generally into 3 categories: Organization barriers; Technical barriers; Social barriers. The 
integration of e-Learning in the education system is likely to gather speed thanks to recent decisions and 
commitment of the Libyan government. Access to ICT facilities is likely to be improved in the very near 
future in all Libyan institutions thanks to major infrastructure projects that are currently in progress. 
However, there is a need for provision of suitable training at different levels, the development of expertise in 
e-Learning use, and research to gather data and inform future developments; these are important factors that 
require plentiful attention and great effort from the Libyan government to ensure the development of 
adequate awareness, attitude, and motivation towards e-Learning as well as suitable responses 
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ABSTRACT 

The paper substantiates inertial nature of the motivation system impact on the individual. Such exposure is a major shift 
from the level of motivational signs of external perception on the level of the individual internal motivation system. This 
approach justifies the ability to control the quality of the individual education as in the process of learning, and at its 
further self-learning. 
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1. INTRODUCTION 

A conversation about human motivation is far from new. 

Existing differences in the vision of causes a particular human response to a specific situation are 
increasingly worse as the accumulation of facts "inexplicable" illogical in terms of existing theories of 
individual behavior. In the educational process understanding of the mechanisms to motivate student learning 
activities, the development of his skills has almost paramount. Inability to understand the reasons for a 
particular activity or, on the contrary, student inactivity makes, in principle, impossible to control the 
educational process, not to mention the ability to manage its self-learning. 

Problems of motivation of educational activities are devoted to E.P. Ilyin [1], Grebenyik O.S., Grebenyik 
T.B. [2], It is an interesting experience in building motivational systems presented by Edward L. Deci, 
Robert J. Vallerand, LUG G. Pelletier, Richard M. Ryan [3], Florian H. Miiller and Marko Palekcic [4]. 


2. THE PROBLEMS OF INDIVIDUAL MOTIVATION 

Today, there are several points of view on the nature motive, E. P. Ilyin identifies the following [1]: motive 
as a need, motive as a target (subject to meet the need), motive as a motivation, motive as an intention, 
motive as resistant properties (personal dispositions), motive as a condition, motive as wording, motive as 
satisfaction. 

Let us talk somewhat more detail on some of them. 

One of the most common points of view on the essence of the motive is statement, according to which the 
first cause, impelling man to perform certain actions, is need. Motive is a conscious motivation, recognized 
need. Needs are a source of human activity, and motive expresses the direction of its activity [2], From this 
perspective, rather convincingly explained the different behavioral patterns, which people are using, getting 
into various life situations. American scientist A. Maslow identified the following levels of human needs, 
placing them on the basis of the relative dominance: physiological needs, safety needs, the need for 
belonging and love, the need for self-esteem and self-actualization need. Distributing thereby subordination 
needs and hence motives. On the basis of the dominance principle proposed - each subsequent need arises 
just as the previous is met. 
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Attempts to explain the irrationality of human behavior led to statements about the motive nature as a 
psychological state of the person. That is to manage human, ie initiation of certain "required" actions - 
motivational system formed involves the creation and consolidation of a certain psychological state in which 
a person can only do so and not otherwise. 

Systematized in the works of E. P. Ilyin point of view, describing the essence of the motive, each in its 
own way, exclude a number of "inexplicable" human behavior facts, leading them to unmotivated actions, 
thus these theories are distinct, not transmitting full perspective. However, if we assume that the issue at hand 
is one, then the above theories are only special cases of motive essence manifestations. 

The author proposes a generalized vision of the motivation nature, built around the concept of the 
development of social inertia subject [5, 6], 


3. THE CONCEPT OF THE SOCIAL SUBJECT INERTIAL 
DEVELOPMENT 

Under the social subject development we will understand the process of different knowledge accumulation, 
skills formation and consolidation of certain skills [6, 7, 8], Or otherwise the development is Knowledge 
accumulation process. Thus, under the Knowledge we will understand a set of specific behavioral individual 
stereotypes describing his response to the news, activity, psychological and other environmental influences. 

Accumulation Knowledge process involves the collection of information, its accumulation and 
understanding that ultimately rearranges or subject to a different quality level of consciousness (brings a new 
basis), or lead to internal conflict, which resulted in the denial, becomes the subject of experience gained 
before (return to original basis). At the same time to estimate the amount of accumulated knowledge, we 
introduce an evaluation criterion - static evaluation, ie something that specific sibject possesses "here and 
now". Besides the static characteristic, the process of subject development is described per dynamic response, 
defined as the degree of subject perception of the new information. Accordingly, the high (increasing) - 
describes the growing interest of the individual to a particular field of knowledge, the average (constant) - is 
the possession of existing knowledge and the lowest (sloping) - extinction of interest to the field of 
knowledge, ie, the experience begins to be used either (a qualitatively new level exit to the new basis), or 
denied (return to the previous basis). The basis will be understood by a set of knowledge and skills is 
essential for the existence of an individual within a given social community (or individual sectors). Changes 
in terms of entity social functioning initiates a change in its baseline, and, conversely, a change of basis of the 
individual outputs it to other social society. 

Thus, the process of acquiring knowledge (development of the subject) can be represented as a movement 
(changing static characteristics) on a trajectory from one basis to another. This movement triggered near both 
internal (goal setting, internal psychological states, the desire for self-expression and self-satisfaction) and 
external (changes in technology, changes in the social environment, changes in the material status, etc.) 
factors [6], 

According to the proposed by the author concept of inertial development of the social subject movement 
from one basis to another, it can be represented by a linear plot of the static evaluation (Accumulated 
knowledge) from time to time (learning, life, ...). Then the transition from one basis to another is macro-aim. 
Macro-aim changing is management resources costly. Much less costly is to make minor adjustments - 
macro-aim formation in the trajectory of the individual, which eventually change the macro-aim structure. 
Visually, you can submit two, built in the form of graphics, time dependencies. One of them is the desired 
path of movement, the other - real characterizing features of this particular individual. Then, setting the 
macro-aim would mean the formation of one or another delta (difference) between the already familiar 
knowledge and the fact that an individual need to learn (to study). Macro-aim setting is a way to control the 
trajectory of the individual and the formation of a stable system of his internal motivation. In other words, for 
an individual to be identified (observable) goals (Macro-aim) and known ways of achieving them. And here 
we are faced with the psychological aspect as interest, n some cases, and even in their majority, the most that 
neither is a promising training programs do not work - only one answer why: they're uninteresting 


410 



International Conference e-Leaming 2014 


4. PERSONALITY PSYCHOLOGY - THE ART OL MANAGEMENT 

Interest is the emotional state associated with the implementation of cognitive activity and this activity is 
characterized by incentive. 

An emotion is a response to the distinction between external and internal factors of the individual 
expectation. Basis, from the point of view of psychology, can be defined as a psychologically comfortable 
condition. Like any physical body or system that seeks to occupy the position which will be characterized by 
the lowest energy consumption, anyone seek to find the most comfortable position, whether it is the basis. 
Any movement that displays an individual of this state requires certain costs. Consequently, the individual 
will be interested in changing their status in only two cases, first, to see if he will gain (consciously or 
unconsciously) the profitability of the other state - another basis, or, second, if there is a clear disadvantage of 
the state in which it is [5, 6], 

Both of these cases are individual path control options, which give an idea about a possible management 
tool. 

Let us return to the concept of interest. In game theory, the greatest interest in the game is evident when 
the player knows the rules of the game and how to influence (control) the game characters: principles of their 
interaction and options of equipment are known (equipment, technical improvements and other services game 
space management options that affect the course of the game), but there is a doubt as a result. Rather, the 
result depends on many small factors that player can evaluate and it is, in principle, a feasible task, but, 
nevertheless, possible inaccuracies of this assessment sharply reduce the predictability of the result of the 
game. 

At the initial stage, the player is faced with uncertainty, it requires some time to learn the rules adopted in 
the game and how to control the impact on the game. At this stage, many of the games cease to be interesting 
- the game is unpredictable, there is no opportunity to evaluate the causes of the failure and, therefore, no 
chance of winning. Too large delta is set- psychologically uncomfortable for a player to distinguish between 
the subjective understanding of the game rules and gaming laws really set. Or on the contrary, the game is 
predictable, the player wins (loses) in any case - the game just becomes uninteresting. In this case, the 
obvious of result negates the need for its refinement and the whole game turns into a process of mechanical 
manipulation of gaming options. 

The game is of interest when all the evidence in game action is present, and game result is still not 
obvious. Work with a" black box. " (Black Box - a device with an unknown principle of work having 
"input" and "output", changing the state of "entry", is necessary to achieve a given reaction to the "exit".) ... 
". For example, the "rules of chess are known in advance. Rules of computer strategy game man "scoops out" 
of his playing experience. He will be losing as long as his ideas about the properties of shapes do not 
coincide with those established by the game creators." [10]. 

Thus, the basic of social subject development trajectory control principle is becoming clear or in a 
particular case management trainee learning trajectory - the formation of a stable system of individual 
internal motivation. Moreover, it becomes obvious, that the fact that playing another similar game, 
involuntarily the player will demonstrate the play behavior stereotypes learnt before Thus explained the 
ability to control not only the individual in the process of moving from one basis to another (in training), but 
also the foundations of its motion task in the future (self-learning control) - the principle of inertia. 

Similar statements can be found in the works of V. K. Tarasov [9]. Individual state it is his picture of the 
world, which identifies goals, objectives and the ways to achieve them are known. The effectiveness of 
training (a special case of an individual path management) will depend on how much change is possible this 
particular world view. 

The first is the connection to the Picture of the world - "catch on the fly." 

"To raise a bucket of water, it is necessary to lean over and take it first". 

Individual movement along a path of development is characterized by the previously constructed internal 
motivation system and any impact on its trajectory will call on his part unconscious (or completely 
conscious) psychological, social or other opposition. Resistance will be stronger with greater corrective 
action. If an individual is initially presented with a structure of adjusted goals, in case of significant 
discrepancies these goals with his expectations, it is likely triggering internal psychological defense, which 
will result in the rejection of all subsequent psychological motivation to achieve the goals presented, even if 
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the profitability of the new approach will be obvious for him. There is an understanding of the goal (there is a 
reason for Action), but there is no movement - psychological state haven't formed yet. 

Therefore, if for any reason the macro-aim is unacceptable to the individual, we should reduce the 
difference between the external (required) and an internal system of objectives - to generate number of 
micro-aims, the achievement of which does not cause a strong enough individual psychological rejection. 
"When we stand on the third step of the common staircase step we can easily step to the fourth step ... to step 
to the sixth is harder, and perhaps it is not possible - to step to the seventh and above" [9], 


5. MAIN RESULTS 

The following results were obtained: 

- provided an overview of the main views on the theory of human motivation; 

- justified inertial nature of the motivation system, which is a major shift from the level of motivational 
signs of external perception on the internal level of individual motivation system; 

- justified the possibility of the learning process quality management through the formation of a trainee 
external motivational system; 

- justified a basis to the subsequent self-learning management graduate, as a manifestation of the learning 
process inertia. 
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ABSTRACT 

One of the new topics taught at technical universities is Internet of Things. In this paper, a workshop for organizing a lab 
in academic environment for the subject Internet of Things is described. The architecture of the platform, scenario and a 
description of components used for creating the environment for learning Internet of things are also given in the paper. 
Students use their knowledge related to creating Android mobile applications and SMS services, as well as given sensors, 
microcontrollers and microcomputers to create an application for smart home automation. 
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1. INTRODUCTION 

Internet of Things (IoT) is a part of the future Internet which consists of billion sensor-based and actuator- 
based smart devices, with data-processing abilities (Perera et al., 2014). According to the Gartner report, over 
26 billion of devices will have been connected to the Internet by the end of 2020 (Gartner, 2013). 

Internet of Things deals with smart homes, as well as other smart environments, such as smart offices, 
smart cities, smart government, smart cites and others (Chan et al., 2008). Smart home enables automation of 
different areas of home, i.e. smart heating, smart air-conditioning, smart window (Jiang et al., 2004). 

In this paper, we propose an environment for teaching Internet of Things at university. A workshop for 
creating applications for smart home automation is realized and a system for smart home automation is 
deployed where students use predefined smart home environments as a framework for creating their own 
applications for automation in order to learn principles of IoT. They are given a scenario and a diagram with 
connections between system components, as well as web services which manage components of the smart 
home. During the workshop, students implement a mobile and SMS application for smart home automation. 


2. ENVIRONMENT FOR TEACHING INTERNET OF THINGS 

An experimental workshop for learning IoT was performed during the laboratory classes of the elective 
subject “Internet of Things” attended by students of the final (fourth) year of the undergraduate studies within 
the e-Business Department, Faculty of Organizational Sciences, University of Belgrade. Eight students 
attended this workshop. The assignment was to create an IoT application for smart home automation. 

2.1 A Scenario for Smart Home Automation 

The main aim of the workshop is to familiarize students with Internet of Things technologies and smart home 
automation. The scenario which is given to students is related to automation of one part of a smart home. 
The scenario includes tracking of sensor status, logic for responding to environmental changes and managing 
particular components of the smart home. Students should manage the air-conditioning system in a smart 
home. If a temperature level is higher than defined threshold, an air-conditioning device should be turned on. 
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The air-conditioning should be turned off if the temperature is below the threshold. For the simulation of air- 
conditioning, a light emitting diode (LED) is used (which is on when the “air-conditioning device” is on). 

2.2 Hardware and Software Infrastructure 

Students are given devices which they control using a given web service. In the Figure 1, a diagram with 
connections between a temperature sensor, LED, Arduino microcontroller and Raspberry Pi microcomputer 
is shown. 



itzlng 


Figure 1 . A diagram of component integration for smart home automation. 

A temperature sensor is connected to Arduino which is connected to Raspberry Pi. A LED which 
simulates air-conditioning is also connected to Raspberry Pi. A web service is running at a web server on 
Raspberry Pi which has a network connection and IP address. 

The web service is REST-based and it enables turning the LED on or off and reading the data from the 
temperature sensor. The web service is connected to scripts written in C++ and Python running on Arduino 
and Raspberry Pi, respectively. 

SMS Cloud platform deployed within the University which is capable of sending and receiving SMS 
messages is used for creating a SMS application for home automation. The SMS platform contains a REST 
API with two functions - for sending and receiving SMSs. First, students need to log into the web application 
for obtaining their own API key. Also, they can obtain their own SMS prefix, which is necessary for 
receiving SMSs. All messages are obtained via the REST API. 

2.3 Deploying Applications for Smart Home Automation 

Students deploy Android mobile application and SMS application for home automation according to the 
given scenario. 

Using the Android application, users can view the current temperature and turn the air-conditioning on or 
off. The final application contains buttons for enabling and disabling the air-conditioning and AUTO button 
which enables or disables the air-conditioning, according to the current temperature level and defined 
threshold. Also, the application can read current temperature from the web service and display the value in 
both Celsius and Fahrenheit degrees. The application logic of AUTO button is deployed on the client side of 
the application. 

The SMS application is based on the SMS Cloud Platform. The SMS application manages the home 
automation by sending SMS messages with particular contents. The application is connected to the web 
service for home automation. For enabling and disabling air-conditioning, a sender should send a message 
with text “prefix on” and “prefix off’, respectively. For getting the information about the current temperature, 
the sender should send a message with text “prefix cel” or “prefix far” for obtaining the current temperature 
in Celsius or Fahrenheit degrees, respectively. For enabling or disabling the air-conditioning, according to the 
temperature threshold, a sender should send a message with text “prefix auto”. The keyword “prefix” should 
be changed with student’s personal prefix. 
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3. CONCLUSION 

This paper describes environment for teaching Internet of Things. It is based on Cloud PaaS (Platform as a 
Service) and hardware layer abstraction which enables using hardware components without knowledge of 
advanced hardware principles. Students generally were satisfied with this workshop. Only one of eight 
participants thought that the workshop had not been interesting. The main problem regarding the workshop 
was the lack in practical knowledge. The majority of participants (7 of 8) thought that this workshop had 
been difficult. 

This environment is a good platform for learning about developing software for Internet of Things 
projects and it provides seamless features for extending applications in the future. After attending the 
workshop, students are able to create their own applications for automation various aspects of smart 
environments. Compared to traditional learning, students are more motivated in adopting IoT concepts by 
using modern technologies in workshops. 
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ABSTRACT 

As a pedagogical approach, problem-based learning (PBL) has shown success for average and gifted students (Hmelo- 
Siver, 2004) and there are numerous incentives for its implementation in online learning environments (Savid-Baden, 
2007; Chemobilsky, Nagarajan, & Hmelo-Silver, 2005). However, little research has been conducted regarding the 
impact of problem-based learning on higher order thinking skills of students with learning disabilities studying in online 
learning environments. This study examines the effects of an online problem-based learning course on critical thinking 
skills of university students with learning disabilities. Students participating in the study will be taking their first course 
in an online Bachelor of Arts degree at the University of Ontario Institute of Technology. Drawing on triangulation, this 
study includes a content analysis of reflective journals, a video analysis of a problem-based learning objective (PBLO) 
and semi-structured interviews with repertory grids, to observe the presence or absence of critical thinking skills among 
students with learning disabilities in an online PBL course. 
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1. INTRODUCTION AND IMPORTANCE OF STUDY 

The 21st century is often referred to as the information based economy and is linked to technological growth, 
accessibility of information, and a need for higher order thinking skills such as problem-solving (Tan, 2007; 
Kalelioglu & Gulbahar, 2013). Dewey (1910) believed long ago that it is impossible to obtain knowledge 
without experience and that rote memorization approaches should be replaced with the development of 
problem-solving skills. Problem-based learning (PBL) has been promoted by numerous researchers 
(Gallagher, Sher, Stepien, & Workman, 1995; Gijbels, Dochy, Van den Bossche, & Siegers, 2005; Hmelo- 
Silver, 2004; Stepien & Gallagher, 1993) as a promising set of strategies that can nurture higher order 
thinking skills like problem solving and critical thinking among average and gifted university students 
(Barrows & Tamblyn, 1980). Recent studies have shown positive connections regarding the implementation 
of instructional strategies like PBL in online learning environments on critical thinking skills (Kalelioglu & 
Gulbahar, 2013) as a way to merge problem-solving skills and technological competencies (Savin-Baden, 
2007). Unfortunately, there has been little research evaluating the influence of PBL as a pedagogical 
approach, attempting to foster critical thinking skills in students with learning disabilities in online learning 
environments. 


2. CONTEXTUAL INFORMATION 

Learning disabilities include a diverse cluster of neurological disorders that can alter the performance of the 
brain in a manner that affects how an individual stores, organizes, understands or uses verbal and non-verbal 
information (LDAC, 2002). The dysfunction in an individual’s brain resulting from the neurological disorder 
causes impairments in one or more processes related to thinking, remembering, perceiving or learning 
(NCLD, 1999). However, in a technologically advanced, global and rapidly changing society of the 21 st 
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century, there is an increased demand for individuals with higher order cognitive skills like problem-solving 
and critical thinking (Behar-Horenstein & Niu, 2011). Critical thinking is defined by Glaser (1942) as an 
outlook and analytical application toward problem solving. Current researchers have further suggested that 
critical thinking is necessary for problem solving, which requires purposeful reflection (Brookfield, 1987; 
Sternberg, 1986; Ennis, 1993; Facione, 1990; Paul, 1997), self-regulation, interpretation, analysis, evaluation 
and inference (Facione, 1990). 

Online problem-based learning has demonstrated effectiveness in enhancing technological, problem 
solving and critical thinking skills (Savin-Baden, 2007). Jonassen, Davidson, Collins, Campbell and Hagg 
(1995) found that the PBL model demands complicated problem solving where learners define the problem, 
identify resources, set priorities, and investigate alternative solutions. These are the same skills and abilities 
necessary for all individuals to employ during real-life problem solving and decision-making activities. If all 
individuals require cognitive skills to be highly adaptable the fast-changing environments of the 21 st century 
as Tan (2007) states, then it is extremely important to ensure students with learning disabilities are also 
achieving these skills. If the goal of education according to Bruner (1960) is to foster the development of 
problem solving skills and encourage the process of inquiry and discussion, then PBL online may be a 
possible approach to take. 


3. METHODOLOGY 

3.1 Research Design and Data Collection Techniques 

This study will examine a set of ethnographic case studies and carry out a triangulation design. “The 
Triangulation Design is a one -phase design in which researchers implement the quantitative and qualitative 
methods during the same timeframe and with equal weight. This design is used when a researcher wants to 
directly compare and contrast quantitative statistical results with qualitative findings or to validate or expand 
quantitative results with qualitative data” (Creswell and Clark, 2007, p. 62). 

Qualitative and quantitative data will be collected through semi-structured interviews incorporating the 
use of repertory grids, content analysis of online discussion boards and video analysis. Observations will be 
recorded and analyzed using a variety of hardware and software in the EILab such as cameras, Adobe 
Connect, Noldus Face Reader and Observer XT. 

3.2 Participants 

The participants of this study will include five students from the online Bachelor of Adult Education and 
Digital Technologies (AEDT) program at the University of Ontario Institute of Technology. The group will 
be partaking in their first online PBL course in the program AEDT1120U Foundations of Digital Teaching 
and Learning Technologies. The five students range in age between twenty and thirty-five and all 
participants have a diagnosed learning disability 

3.3 Research Questions 

1. Can problem-based learning online foster critical thinking skills in students with learning disabilities? 

2. Can problem-based learning online foster engagement in students with learning disabilities? 

3. Does an online PBL environment support principals of Universal Design for Learning (UDL)? 


4. CONCLUSIONS 

The purpose of this paper is to discuss the preliminary research design of a study that will aim to analyze the 
presence or absence of higher order thinking skills demonstrated by students with learning disabilities in an 
online learning environment. Other areas of interest will focus on the level of engagement present in an 
online PBL learning environment and a gap analysis of Universal Design for Learning principals. 
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A SYSTEM FOR THE AUTOMATIC ASSEMBLY OF 
TEST QUESTIONS USING A NO-SQL DATABASE 
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ABSTRACT 

We describe a system that automatically assembles test questions from a set of examples. Our system can create test 
questions appropriate for each user’s level at low cost. In particular, when a user review their lesson, our system provides 
new test questions which are assembled based on their previous test results and past mistakes, rather than a completely 
random selection. By providing test question of the appropriate level, we hope to enhance the user’s learning. We use 
MongoDB which is a no-SQL database system because the contents (test questions) are not structured. Most of test 
contents used in class are not structured, i.e. each question has a different structure and different selection type. Therefore, 
we used MongoDB system in order to structure the test questions more freely. 


KEYWORDS 

e-learning, personalization, automatically assembling, non-structure. 


1. INTRODUCTION 

In a class such as the language course of a university, the level of individual students will vary, even if 
classes are divided based on student ability. Moreover, even if students begin a course at the same level, 
differences in level will emerge as the course proceeds as students progress at different rates. However, until 
now, it has been common to present the same test questions to every student in the class. Even if, ideally, 
questions should be matched with each student's level, this has been impractical. Some part of this problem 
can be addressed with the growth of latest e-learning technology (Xia et al, 2014; Kundisch et al, 2013). 

First, by using an e-learning system, each student's test result can be evaluated in real time by using the 
network. Second, the results for each student can be compared immediately. Third, by comparison of a 
student’s test result with those of other students, the level of the student relative to the class can be assessed. 
In this way, if a study result is known in real time, it is possible to provide the test questions matched with 
each student at the next step. 

Our system separates the example answers from the question. All example answers are managed in a 
database, with the relation information relevant to the questions. The sentences used for test questions are 
reconstructed using many example answers managed in the database. The database also has information 
about the results of each student for each question and example answers. 


2. OUR SYSTEM FOR AUTOMATICALLY ASSEMBLING TEST 
QUESTIONS 


The goal of our system is the provision of the test questions matched to the level of each student. After the 
first test about their lesson, we collect the information about their test result. This information is used to 
restructure the test questions when the student restudies about their past test or lesson. The construction of a 
system that allows students to improve their learning by solving specially selected new test questions is the 
concept of our system. 
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The common test questions have various forms. Some questions are multiple choice and some questions 
require a written answer. The various types of questions make it difficult when data is stored in a relational 
database system. Moreover, since the most test questions were created by a program like a Microsoft Word, 
they cannot be stored in a database as-is. Data in MongoDB has a flexible schema. Collections do not enforce 
document structure (Kanade et al. 2014; Banker, 2011). This flexibility allows our system to control the 
document-oriented data. 


3. SYSTEM ARCHITECTURE 

For many, but not all, test questions, example answers can be separated from the question. We call each 
example answer or question “fragment data”. We consider those questions from which example answers can 
be separated and focus on how to store the question and example answers from which new questions related 
to the original question can be formed. An initial set of questions was compiled by foreign language teachers. 
These questions were categorized by our system according to whether example answers can be separated or 
not. 

A teacher who wants to use a test question can use the question as it is or a new question can be generated 
using the stored example answers. Our system supports the reassembly of the example answers with their 
related question automatically if the question has many example answers that can reassemble. 

When our system provides the reassembled test question, our system refers to the percentage of correct 
answers for each test question and example answers and provides the test questions appropriate for the level 
of each student based on feedback of the student’s previous results. 


Teacher A 


Make contents 

20% 

Make contents 


40% 

/ 


Questions and Related Example Answers 


Student A 


Teacher B 

65%(Percentage of correct answers) 


Teacher C 


5093^ Contents 
-'k% Server 



feedback 


45% '' v 

feedback "* 


Test question for student A 


Student B 

Test question for student B 


Figure 1 . Architecture of System 

Figure 2 shows a very simple sample data structure. The provided test question (The data matched to the 
level of student in Figure 2.) is constructed from the stored example answers (Example answers in Figure 2.). 
Each example answer has the percentage of correct answers (46%~78%). This percentage is used to generate 
questions of an appropriate level for each student, based on the student’s previous test results. Of course, this 
percentage is changed by re-collection of test results. 

Moreover, one test question is stored as one JSON style document, with various data added to each 
document. 
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Example answers 


The data matched to the level of student 


1 

_id: <AsSecProLevlld2386> ; 
level : 1, 

group: "pronounce", 
pro: "ow" 

As:{ 

grow: 60 
brow: 55 
power: 70 
prow: 62 
cow: 78 
brown: 54 
owl: 46 
} 

} 



Database 


{ 

_id: <QtSecProLevlld87>, 
userid: "testlOOl", 
qlevel : 1, 
group: pronounce, 
type : "select", 
group_ans: 65 
a ns: "y" 

questionjd: "QtSecProLevlld87", 
examjd: "AsSecProLevlld2386" 
examples: { 
grow: 60 
brow: 55 
power: 70 
prow: 62 

i 


Test question 


Web page 


Figure 2. Example of Data Structure 


4. CONCLUSION 

We proposed a system for automatically assembling test questions. This system can assemble the example 
answers in the test questions automatically and at low cost. The processing of flexible data became possible 
by using MongoDB. 
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